Hosted Kubernetes clusters

Overview

Hosted Kubernetes is designed to make it easier for you to manage and orchestrate workloads on your Mithril compute. We provide a long-lived control plane instance which hosts the Kubernetes API server, and allows you to toggle joining one of these Kubernetes clusters when you place a spot bid.

• Upon placing a spot bid with a Kubernetes cluster selected, Mithril will automatically join the new instances to the Kubernetes cluster on startup.

• When preemption happens, Mithril sends a drain command to the affected instances, starting a 5 minute timer for them to gracefully shutdown. After the time is up, Mithril powers off the instance.

• If the instance gets reallocated, Mithril sends an uncordon command to the instance so Kubernetes may resume scheduling on it once it is running again.

• When a spot bid is terminated (and therefore the instances permanently deleted), Mithril deletes them from the Kubernetes Cluster.

Attaching persistent storage

To make persistent storage volumes accessible within the k8s cluster:

1. Mount the volume to all nodes in the cluster for a homogeneous setup. You select which volumes to mount when placing your bid, or when the bid is paused.

2. Use hostPath for pod mount, e.g:

volumes:
  - name: data-volume
    hostPath:
      path: /mnt/your-fileshare-name
      type: Directory

FAQ

What comes installed with the Kubernetes cluster?

Hosted Kubernetes clusters come pre-loaded with Cilium CNI and NVIDIA GPU Operator, to make setup easier. Users are free to adjust these defaults and modify the cluster configuration however they'd like. This is not a fully "managed" offering, Foundry does not manage updates and settings beyond adding/cordoning/removing nodes from the cluster as they go through the Spot lifecycle.

Can the Kubernetes clusters be accessed from outside the Mithril?

Yes. The control plane comes with a public IP that has the SSH port (22) and Kubernetes API port (6443) open by default.

Does Mithril Hosted Kubernetes support autoscaling?

You can configure autoscaling for deployments in Kubernetes, but currently there is no officially supported mechanism for autoscaling the underlying cluster size. We recommend using the Foundry Spot API to programmatically place/modify spot bids, for custom autoscaling.