Skip to main content

How to provision a GPU on a VM

Hikube allows you to attach one or more NVIDIA GPUs directly to a virtual machine. This guide explains how to choose the GPU type suited to your workload, create a VM with GPU, and verify that hardware acceleration is available.

Prerequisites​

  • kubectl configured with your Hikube kubeconfig
  • SSH access to the VM (public SSH key available)
  • Familiarity with virtual machine concepts on Hikube

Steps​

1. Choose the GPU type​

Hikube offers several NVIDIA GPUs suited to different use cases:

GPUArchitectureMemoryUse case
L40SAda Lovelace48 GB GDDR6Inference, development, prototyping
A100 (PCIe / SXM4)Ampere80 GB HBM2eML training, fine-tuning
RTX PRO 6000 BlackwellBlackwell96 GB GDDR7LLM, intensive computing, distributed training
Which GPU to choose?

Start with an L40S for development and prototyping. Move to an A100 for standard ML model training, and reserve the RTX PRO 6000 (Blackwell) for the most demanding workloads such as LLM training or high-performance computing.

The GPU identifiers to use in your manifests are:

GPUgpus[].name value
L40Snvidia.com/AD102GL_L40S
A100 PCIe 80 GBnvidia.com/GA100_A100_PCIE_80GB
A100 SXM4 80 GBnvidia.com/GA100_A100_SXM4_80GB
RTX PRO 6000 Blackwellnvidia.com/GB202GL_RTX_PRO_6000_BLACKWELL_SERVER_EDITION

2. Create the VM manifest with GPU​

Create a manifest that declares the desired GPU in the spec.gpus section:

gpu-vm.yaml
apiVersion: apps.cozystack.io/v1alpha1
kind: VMDisk
metadata:
  name: gpu-workstation-disk
spec:
  source:
    image:
      name: ubuntu-2404
  storage: 100Gi
  storageClass: replicated
---
apiVersion: apps.cozystack.io/v1alpha1
kind: VMInstance
metadata:
  name: gpu-workstation
spec:
  runStrategy: Always
  instanceProfile: ubuntu
  instanceType: u1.2xlarge
  gpus:
    - name: "nvidia.com/AD102GL_L40S"
  disks:
    - name: gpu-workstation-disk
  external: true
  externalMethod: PortList
  externalPorts:
    - 22
    - 8888
  sshKeys:
    - ssh-ed25519 AAAA... user@host
Recommended CPU/GPU ratio

Plan for 8 to 16 vCPUs per GPU. For a single GPU, a u1.2xlarge (8 vCPU, 32 GB RAM) is a good starting point. For multi-GPU, scale up to u1.4xlarge or u1.8xlarge.

3. Deploy the VM​

Apply the manifest:

kubectl apply -f gpu-vm.yaml

Wait for the VM to reach the Running state:

kubectl get vminstance gpu-workstation -w

Expected output:

NAME               STATUS    AGE
gpu-workstation    Running   2m
note

Provisioning a VM with GPU may take a few extra minutes compared to a standard VM, as the GPU needs to be allocated and attached.

4. Verify the GPU in the VM​

Connect to the VM via SSH:

virtctl ssh -i ~/.ssh/id_ed25519 ubuntu@gpu-workstation

Verify that the GPU is detected:

nvidia-smi

Expected output:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 535.xx.xx    Driver Version: 535.xx.xx    CUDA Version: 12.x     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  NVIDIA L40S         Off  | 00000000:00:06.0 Off |                    0 |
| N/A   30C    P0    35W / 350W |      0MiB / 46068MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

For detailed information:

nvidia-smi --query-gpu=name,memory.total,utilization.gpu --format=csv

5. Configure a multi-GPU VM​

For intensive workloads (distributed training, large-scale inference), you can attach multiple GPUs to a single VM by repeating entries in spec.gpus:

multi-gpu-vm.yaml
apiVersion: apps.cozystack.io/v1alpha1
kind: VMDisk
metadata:
  name: multi-gpu-disk
spec:
  source:
    image:
      name: ubuntu-2404
  storage: 200Gi
  storageClass: replicated
---
apiVersion: apps.cozystack.io/v1alpha1
kind: VMInstance
metadata:
  name: multi-gpu-workstation
spec:
  runStrategy: Always
  instanceProfile: ubuntu
  instanceType: u1.8xlarge
  gpus:
    - name: "nvidia.com/GA100_A100_SXM4_80GB"
    - name: "nvidia.com/GA100_A100_SXM4_80GB"
    - name: "nvidia.com/GA100_A100_SXM4_80GB"
    - name: "nvidia.com/GA100_A100_SXM4_80GB"
  disks:
    - name: multi-gpu-disk
  external: true
  externalMethod: PortList
  externalPorts:
    - 22
    - 8888
  sshKeys:
    - ssh-ed25519 AAAA... user@host
warning

For multi-GPU, size the instance type accordingly. Plan for at least 8 vCPUs and 32 GB of RAM per GPU. A u1.8xlarge (32 vCPU, 128 GB RAM) is suitable for 4 GPUs.

Verification​

After deployment, confirm that everything is working:

  1. Check the VM status:
kubectl get vminstance gpu-workstation
  1. Check the GPU in the VM:
virtctl ssh -i ~/.ssh/id_ed25519 ubuntu@gpu-workstation -- nvidia-smi
  1. Test CUDA (if drivers are installed):
virtctl ssh -i ~/.ssh/id_ed25519 ubuntu@gpu-workstation -- nvidia-smi --query-gpu=name,memory.total,driver_version,cuda_version --format=csv,noheader

Next steps​