GPU CUDA: run CUDA workloads against a host GPU

Runs CUDA compute workloads inside a smolvm microVM against a real host NVIDIA GPU, using smolvm’s —cuda API remoting. Use when a workload in a machine needs a GPU; when nvidia-smi or /dev/nvidia* is missing inside a —cuda guest; when a CUDA program in a machine exits zero but seems not to touch the device; when the shim will not load in an Alpine image; or when checking whether CUDA is available on a given platform, including Windows. Do not use it for Vulkan graphics (—gpu), which is a separate feature, in docs/gpu.md, and do not expect it on a Mac, which has no CUDA driver.

Verified on smolvm v1.34.0 against an NVIDIA A10 (driver 580.105.08, Linux x86_64, kernel 6.8.0-1064-nvidia), 2026-11-09, and on v1.22.2 against an NVIDIA GeForce RTX 4050 (driver 32.0.15.6626, Windows x86_64), 2026-10-03. Done means a program in the VM opens the device, creates a context, and moves data to and from it. The Linux runs used the scripts of their date; this version’s preflight and cleanup scripts ran on Linux aarch64 on v1.22.2 on 2026-10-03.

The GPU path was last run end to end on Linux x86_64 on v1.34.0, on an A10: the preflight, the probe against two glibc images and the cleanup, twice; the three eval prompts last ran on v1.14.6. Windows was re-run on v1.22.2 against an RTX 4050: the probe, the two absences, the shim in a plain alpine guest, and a vector add over 16777216 elements; see references/windows.md. The macOS arm64 and Linux aarch64 hosts have no NVIDIA hardware, so what runs there is the no-GPU answer, on v1.22.2 on macOS and v1.18.2 on Linux aarch64: the preflight reports gpu_present=no without starting anything, and --cuda reaches a CPU emulation device, which the probe names rather than passing. “What was not run” lists every remaining step.

How this works, and why the checks are shaped this way

The guest gets no NVIDIA driver and no /dev/nvidia*. A compatibility libcuda.so.1 is injected at /opt/smolvm-cuda inside the guest and forwards CUDA driver calls over vsock to a host daemon that owns the device. Nothing is needed on the host beyond a working NVIDIA driver: no extra packages, no container toolkit, no device plugin.

Because the API is remoted rather than passed through, a program can start, link against libcuda.so.1 and exit zero without a GPU ever being reached. Assert a device name and a transferred result, never an exit code.

Procedure

1. Preflight.

scripts/preflight.sh

Read-only: it starts no VM and touches no NVIDIA state. It reports the GPU and driver version, whether your user can open /dev/kvm, and the host’s own libcuda count. If the question is “can this machine run CUDA”, the preflight answers it and nothing else needs to run. result=blocked with gpu_present=no saves the most time, because on a host with no NVIDIA GPU the probe starts a VM, reaches a CPU emulation device and returns a passing round trip, which reads like a yes. Verified on macOS arm64 on v1.16.1, v1.18.2 and v1.22.2: gpu_present=no and unsupported=cuda; on v1.22.2 the note read macOS has no CUDA driver, so --cuda has nothing to reach here.

2. Run the probe.

scripts/run-cuda-probe.sh                     # python:3.12-slim
scripts/run-cuda-probe.sh <other glibc image>

It mounts scripts/ into the guest and runs cuda-probe.py under --cuda, then asserts two values from the output:

device_named=ok
data_roundtrip=ok
device_kind=gpu
result=cuda_ok

On the A10 on v1.14.6, cuda-probe.py’s own output was:

load_shim -> ok /opt/smolvm-cuda/libcuda.so.1
cuInit -> 0
cuDeviceGetCount -> 0 count = 1
cuDeviceGetName -> 0 name = NVIDIA A10
cuCtxCreate -> 0
cuMemGetInfo -> 0 total MiB = 22587
cuMemAlloc   -> 0
cuMemcpyHtoD -> 0
cuMemcpyDtoH -> 0
roundtrip first 16 bytes match: True
cuMemFree    -> 0

roundtrip ... True is the one that matters. It is the only line that proves bytes reached the device.

3. Clean up. CUDA images are large.

scripts/cleanup.sh --purge
smolvm machine prune --name <NAME> --all   # any persistent --cuda machine you kept

cleanup.sh waits up to 20 seconds, polling the machine list, and prints waiting=up to 20s first: an ephemeral machine’s entry retires after its run returns.

The probe runs are ephemeral and leave nothing to prune; machine prune takes a machine name and is rejected without one.

--cuda changes nothing on the host: the shim is injected inside the guest only. Verified after a series of CUDA runs plus a Kubernetes install and teardown on the same host, where nvidia-smi still reported the device and a whole-filesystem sweep for *smolvm* came back empty.

Traps

Full detail in references/traps.md.

  • A zero exit code proves nothing. The remoted API is why.
  • A device name and a passing round trip do not prove a GPU either. When the host cannot load the NVIDIA driver library, the host side answers with a CPU emulation device: cuInit -> 0, cuDeviceGetCount -> 0 count = 1, cuDeviceGetName -> 0 name = smolvm CPU emulation device, total MiB = 1024, and roundtrip first 16 bytes match: True. The device name is the discriminator. scripts/run-cuda-probe.sh now reports device_kind=cpu_emulation and result=cpu_emulation_not_gpu for it instead of result=cuda_ok.
  • nvidia-smi is absent inside the guest and that is correct. So is /dev/nvidia*. Neither is a useful check.
  • Use a glibc image and load the shim by absolute path. The shim is glibc, so an Alpine guest cannot load it, and relying on the loader path picks up whatever the image carries.
  • A fresh GPU cloud instance does not have KVM access for your user, and the installer says the install succeeded anyway. sg kvm -c applies the group without a logout.
  • agent did not become ready within 30 seconds from the probe is about memory, not the GPU. The probe asks for 8192 MiB. On v1.18.2 a Linux aarch64 host with no NVIDIA GPU failed that way, reached the CPU emulation device with the same probe at 1024 MiB, and failed the same way on a plain machine run --mem 8192 with no --cuda at all.
  • --cuda and --gpu are different features. --cuda is compute over vsock; --gpu is Vulkan over virtio-gpu, which reached a Venus device on macOS arm64 on v1.18.2 and is covered by docs/gpu.md in the repository. On Windows, on v1.14.6 and v1.22.2, --gpu boots and the guest has no GPU device; nothing reports that the flag was dropped (references/windows.md).

Security defaults, and why they are the defaults

  • The guest never gets the device, and that is the isolation. No /dev/nvidia* is passed through, so a workload in the machine cannot reach the driver directly, reprogram it, or see another VM’s device state through it. What it gets is a forwarded API surface.
  • What that surface exposes is still real. A remoted CUDA call runs against the host’s driver and the host’s memory allocator, so treat a --cuda machine as having a channel to a privileged host component. The VM boundary is what makes that acceptable; it is not zero authority.
  • Nothing here needs root or a container toolkit on the host. If a procedure asks you to install a device plugin or run the CLI as root to get CUDA working, it is not this procedure.
  • The scripts wrap the public CLI only, mount only this packet’s own scripts/ directory read-only into the guest, and cleanup deletes only names it recorded under the smolskill- prefix.

Platform arms

  • Linux x86_64 with an NVIDIA GPU: verified on an A10 on v1.34.0 (driver 580.105.08, kernel 6.8.0-1064-nvidia) and on v1.14.6, preflight through cleanup, with the probe run against two glibc images.
  • Windows x86_64 with an NVIDIA GPU: verified on an RTX 4050 on v1.22.2, on 2026-10-03, and on v1.14.6 on 2026-09-11 (driver 32.0.15.6626), including the device name and a 1 MiB device round trip. references/windows.md.
  • macOS arm64 and Intel: not applicable. macOS has no CUDA driver, so --cuda has nothing to reach. The preflight says so rather than letting a run time out.
  • Linux aarch64: no NVIDIA hardware on the host tested. The preflight and the probe against the CPU emulation device were run on v1.18.2. On v1.22.2 the probe timed out on that host’s memory before reaching the device.
  • Multi-GPU, branches of a --cuda machine, machines restored from its checkpoint, and a training or inference workload: not run anywhere.

Eval prompts, and what they produced

1. “Run a CUDA workload in a smolvm machine and prove it reached the GPU.” (re-run on v1.14.6 on the A10)

cuda-probe.py’s output is the block under step 2 above, from this run.

2. ”nvidia-smi is not in my --cuda guest and there is no /dev/nvidia0. Is the GPU working?” (re-run on v1.14.6 on the A10)

Both absences are correct. Inside a --cuda guest on python:3.12-slim:

--- /opt/smolvm-cuda ---
libcublas.so.11
libcublas.so.12
libcublas.so.13
libcublasLt.so.11
libcublasLt.so.12
libcublasLt.so.13
libcuda.so
libcuda.so.1
--- /dev/nvidia* ---
ls: cannot access '/dev/nvidia*': No such file or directory
--- nvidia-smi ---
nvidia-smi: not present in the image
--- SMOLVM_CUDA env ---
SMOLVM_CUDA_ZEROCOPY=1

The driver API is the check, and the probe above is what runs it.

3. “Can this machine run CUDA?”

On the A10 on v1.14.6, where the answer is yes:

platform=linux-x86_64
accel=kvm
accel_access=ok
gpu_present=yes
gpu=NVIDIA A10, 580.105.08
host_libcuda=1
guest_needs_glibc_image=yes
guest_shim_path=/opt/smolvm-cuda/libcuda.so.1
result=ready

And where it is no:

macOS 27.0.1 arm64 on v1.22.2:

platform=darwin-aarch64
gpu_present=no
unsupported=cuda
note=macOS has no CUDA driver, so --cuda has nothing to reach here.
result=blocked

Lima linux-kvm, Ubuntu 24.04 aarch64, on v1.22.2:

platform=linux-aarch64
accel_access=ok
gpu_present=no
note=no nvidia-smi on this host; --cuda needs the driver library libcuda.so.1, which host_libcuda reports
host_libcuda=0
note=libcuda.so.1 is not in this host's loader cache; --cuda loads it on the host, and without it the guest reaches the CPU emulation device
result=blocked

Re-verified on v1.34.0, the GPU path

Run 2026-11-09 PT against v1.34.0 from the published release, checksum checked, under fresh isolated HOMEs on an NVIDIA A10 cloud instance (Ubuntu 22.04 x86_64, 30 vCPU Intel Xeon Platinum 8358, 222 GiB, kernel 6.8.0-1064-nvidia, driver 580.105.08), twice, the second time from a fresh HOME. The preflight said gpu_present=yes, host_libcuda=1 and result=ready. The probe on python:3.12-slim in both passes, and once on python:3.12-bookworm, printed name = NVIDIA A10, total MiB = 22587 and roundtrip first 16 bytes match: True, and reported device_named=ok, data_roundtrip=ok, device_kind=gpu and result=cuda_ok. The cleanup left machines=clean and vm_processes=none, and afterwards nvidia-smi reported 0 MiB used. The eval prompts were not re-run.

Re-verified on v1.22.2, the no-GPU path

Run 2026-10-03 PT against v1.22.2 from the published release, under an isolated HOME on macOS 27.0.1 arm64, twice, the second time from a fresh HOME. The preflight said gpu_present=no and result=blocked without starting anything. The probe reached the emulation device, name = smolvm CPU emulation device and roundtrip first 16 bytes match: True, and reported device_kind=cpu_emulation and result=cpu_emulation_not_gpu. In the second pass Docker Hub refused the default image with TOOMANYREQUESTS, the anonymous pull limit, and the probe passed on scripts/run-cuda-probe.sh mirror.gcr.io/library/python:3.12-slim, the same image from a mirror. A probe that reaches no device now says device_kind=none; earlier versions of the script said gpu.

On Lima linux-kvm (Ubuntu 24.04 aarch64) the probe’s 8192 MiB guest timed out on a host that booted 2048 MiB and no more on 2026-10-03, and the script named that cause first. Linux aarch64 was last verified on v1.18.2, where the probe at --mem 1024 reached the emulation device with the same lines as macOS.

Re-verified on v1.14.6

Run 2026-09-11 PT against v1.14.6 from the published release, installed into an isolated data root on an NVIDIA A10 cloud instance: 30 vCPU Intel Xeon Platinum 8358, 222 GiB memory, kernel 6.8.0-1046-nvidia, driver 580.105.08, 23028 MiB of device memory.

The GPU path ran end to end on this release. preflight, probe, all three eval prompts and cleanup were executed in the packet’s own order:

preflight            result=ready, gpu_present=yes, gpu=NVIDIA A10, 580.105.08, host_libcuda=1
probe, default image device_named=ok, data_roundtrip=ok, result=cuda_ok
probe, second image  device_named=ok, data_roundtrip=ok, result=cuda_ok
cleanup --purge      machines=clean, vm_processes=none

The probe was run against two glibc images, python:3.12-slim and python:3.12-bookworm, and both reported name = NVIDIA A10 and total MiB = 22587, and the first 16 bytes of the 1 MiB round trip came back as sent.

The host is untouched by --cuda, and that is now measured rather than quoted. After these runs nvidia-smi reported NVIDIA A10, 580.105.08, 23028 MiB, 0 MiB used, a whole-filesystem sweep for *smolvm* outside the install prefix and the data root returned nothing, and the eight NVIDIA kernel modules were still loaded.

The KVM precondition in the traps reproduced on this instance: a fresh instance has /dev/kvm as root:kvm with the login user outside the group, so the user could not open it until the group was granted; the preflight, run after that, reported accel_access=ok.

What was not run

  • The preflight and the cleanup on Windows. scripts/*.sh are bash and do not run there, so the Windows runs issued the probe and the eval prompts by hand.
  • --cuda with a CUDA base image (nvidia/cuda:...), and the apt-get install -y python3 those images need. Both probe runs used a Python image, which is what the packet recommends.
  • The sg kvm -c single-command remedy. The state it addresses, a user who cannot open /dev/kvm, did reproduce on a fresh instance; the fix applied was a group change and a new login session.

Not run on any platform or release:

  • Branches and restores of a --cuda machine. The docs site’s introduction/concepts/gpu.md (smolmachines.com/docs) gives this as a reason for the remoting design, and a restored CUDA machine has its own shorter 10 s ready timeout.
  • Multi-GPU, and contention between two machines sharing one device.
  • A training or inference workload. The furthest run is the vector add on Windows; the rest are driver-API assertions, and say nothing about how much of the CUDA API surface is implemented.
  • install for the KVM group precondition, which a fresh GPU instance fails.
  • teardown for the cleanup script. CUDA images are large enough that machine prune is worth it.

Scripts

The files the procedure runs, in the order it runs them. It calls each one by the path in its heading, relative to the directory the procedure is saved in.

scripts/preflight.sh

#!/usr/bin/env bash
# Report whether this host can run CUDA workloads inside a smolvm machine.
# Read-only: starts no VM, touches no NVIDIA state, changes no group membership.
#
# Output is one key=value per line. The last line is always result=ready or
# result=blocked.

set -uo pipefail

# The Linux GPU path last ran on 1.34.0; the no-GPU answer on macOS on 1.22.2.
case "$(uname -s)" in Linux) VERIFIED_VERSION="1.34.0" ;; *) VERIFIED_VERSION="1.22.2" ;; esac

emit() { printf '%s=%s\n' "$1" "$2"; }
note() { printf 'note=%s\n' "$1"; }

blocked=0

SMOLVM="${SMOLVM:-$(command -v smolvm 2>/dev/null)}"
if [ -z "$SMOLVM" ]; then
    emit smolvm_installed no
    blocked=1
else
    emit smolvm_installed yes
    version="$("$SMOLVM" --version 2>/dev/null | awk '{print $NF}')"
    emit smolvm_version "${version:-unknown}"
fi

emit verified_version "$VERIFIED_VERSION"
if [ -n "${version:-}" ] && [ "$version" != "unknown" ]; then
    if [ "$version" = "$VERIFIED_VERSION" ]; then
        emit version_status match
    else
        newest="$(printf '%s\n%s\n' "$version" "$VERIFIED_VERSION" | sort -V | tail -1)"
        if [ "$newest" = "$version" ]; then
            emit version_status newer
            note "this packet was verified on $VERIFIED_VERSION on this platform and the binary is $version, so check each step's output against the binary"
        else
            emit version_status older
        fi
    fi
else
    emit version_status unknown
fi

kernel="$(uname -s)"
arch="$(uname -m)"
case "$arch" in aarch64|arm64) arch=aarch64 ;; esac
emit platform "$(printf '%s' "$kernel" | tr '[:upper:]' '[:lower:]')-$arch"

case "$kernel" in
    Darwin)
        emit accel hvf
        if [ "$(sysctl -n kern.hv_support 2>/dev/null)" = "1" ]; then emit accel_access ok; else emit accel_access denied; fi
        emit gpu_present no
        emit unsupported "cuda"
        blocked=1
        note "macOS has no CUDA driver, so --cuda has nothing to reach here."
        ;;
    Linux)
        emit accel kvm
        # A fresh cloud GPU instance fails this, and the installer warns and
        # continues, so the install succeeds and the first VM fails.
        if [ -r /dev/kvm ] && [ -w /dev/kvm ]; then
            emit accel_access ok
        else
            emit accel_access denied
            blocked=1
            note "your user cannot open /dev/kvm. Fix without logging out: sudo usermod -aG kvm \$USER, then run the next command through sg kvm -c '...'"
        fi
        ;;
    *)
        emit accel unknown
        emit accel_access unknown
        note "this script covers macOS and Linux. CUDA was verified on Windows by hand on v1.22.2; see references/windows.md."
        blocked=1
        ;;
esac

if command -v nvidia-smi >/dev/null 2>&1; then
    gpu="$(nvidia-smi --query-gpu=name,driver_version --format=csv,noheader 2>/dev/null | head -1)"
    if [ -n "$gpu" ]; then
        emit gpu_present yes
        emit gpu "$gpu"
    else
        emit gpu_present no
        blocked=1
        note "nvidia-smi is installed but reported no device"
    fi
elif [ "$kernel" = "Linux" ]; then
    emit gpu_present no
    blocked=1
    note "no nvidia-smi on this host; --cuda needs the driver library libcuda.so.1, which host_libcuda reports"
fi

# The host's own libcuda is what the remoting daemon forwards to. The guest gets
# none, and that is the design. ldconfig is often in /sbin, off a user's PATH.
if [ "$kernel" = "Linux" ]; then
    ldc="$(command -v ldconfig 2>/dev/null)"
    for c in /sbin/ldconfig /usr/sbin/ldconfig; do [ -n "$ldc" ] || { [ -x "$c" ] && ldc="$c"; }; done
    if [ -n "$ldc" ] && cache="$("$ldc" -p 2>/dev/null)"; then
        host_libcuda="$(grep -c 'libcuda\.so\.1' <<<"$cache")"
        emit host_libcuda "$host_libcuda"
        if [ "$host_libcuda" -eq 0 ]; then
            blocked=1
            note "libcuda.so.1 is not in this host's loader cache; --cuda loads it on the host, and without it the guest reaches the CPU emulation device"
        fi
    else
        emit host_libcuda unknown
        note "could not read the loader cache with ldconfig -p, so whether libcuda.so.1 loads was not checked"
    fi
fi

emit guest_needs_glibc_image yes
emit guest_shim_path /opt/smolvm-cuda/libcuda.so.1

if [ "$blocked" -eq 0 ]; then emit result ready; else emit result blocked; fi

scripts/run-cuda-probe.sh

#!/usr/bin/env bash
# Run the CUDA driver-API probe inside a --cuda machine and assert its values.
#
# usage: run-cuda-probe.sh [<image>]      (default python:3.12-slim)
#
# The image must be glibc. The injected shim is glibc, so an Alpine (musl) guest
# cannot load it. python:3.12-slim is small and already has Python, which CUDA
# base images do not.

set -uo pipefail

here="$(cd "$(dirname "$0")" && pwd)"
IMAGE="${1:-python:3.12-slim}"

SMOLVM="${SMOLVM:-$(command -v smolvm 2>/dev/null)}"
if [ -z "$SMOLVM" ]; then
    printf 'smolvm not found; set SMOLVM to its path\n' >&2
    exit 2
fi

out="$("$SMOLVM" machine run --cuda --net --mem 8192 \
    -v "$here:/probe:ro" --image "$IMAGE" -- python3 /probe/cuda-probe.py 2>&1)"
printf '%s\n' "$out" | sed 's/^/  /'

fail=0
check() {
    if printf '%s' "$out" | grep -q "$2"; then
        printf '%s=ok\n' "$1"
    else
        printf '%s=FAIL\n' "$1"
        fail=1
    fi
}

# A device NAME, not a zero return code: on a remoted API a program links and
# exits zero without a GPU ever being reached.
check device_named 'cuDeviceGetName -> 0 name = ..*'
# And the round trip, which is the only proof that bytes reached the device.
check data_roundtrip 'roundtrip first 16 bytes match: True'

# A host that cannot load the NVIDIA driver library answers with a CPU emulation device, which
# passes both checks above. The device name is what tells them apart.
if printf '%s' "$out" | grep -qi 'name = .*emulation'; then
    printf 'device_kind=cpu_emulation\n'
    printf 'result=cpu_emulation_not_gpu\n'
    printf 'The guest reached a device and the round trip returned, but the device is\n'
    printf 'smolvm CPU emulation, not an NVIDIA GPU. Nothing here was accelerated. Run\n'
    printf 'scripts/preflight.sh: gpu_present=no says the same thing without starting a VM.\n'
    exit 1
fi
# A run that reached no device at all names none, so it must not read as a GPU.
if printf '%s' "$out" | grep -q 'cuDeviceGetName -> 0 name = ..*'; then printf 'device_kind=gpu\n'; else printf 'device_kind=none\n'; fi

if [ "$fail" -eq 0 ]; then
    printf 'result=cuda_ok\n'
else
    printf 'result=FAILED\n'
    printf 'Run scripts/preflight.sh. The three causes, and none of them says so in the error:\n'
    printf '  - A guest too large for this host. The probe asks for 8192 MiB, and a host that cannot\n'
    printf '    boot that inside the fixed 30 s fails with "agent did not become ready within 30\n'
    printf '    seconds". Run smolvm machine run --mem 8192 --net --image alpine -- true: if that\n'
    printf '    fails the same way, it is the host, not CUDA.\n'
    printf '  - A musl image. The injected shim is glibc, so an Alpine guest cannot load it.\n'
    printf '  - A machine started without --cuda, in which case /opt/smolvm-cuda does not exist.\n'
fi
exit "$fail"

scripts/cuda-probe.py

#!/usr/bin/env python3
"""Prove a smolvm --cuda guest reaches a real GPU, and that data moves.

Run this inside the guest. It uses the CUDA driver API through ctypes, so it
needs no CUDA toolkit, no torch and no nvidia-smi.

Why it is shaped this way. CUDA in smolvm is REMOTED, not passed through: the
guest gets no NVIDIA driver and no /dev/nvidia*, and a compatibility
libcuda.so.1 forwards driver calls over vsock to a host daemon that owns the
device. A program can therefore start, link against libcuda.so.1 and exit zero
without a GPU ever being reached, so an exit code proves nothing. The two
assertions that mean something are a device NAME and a round trip through
device memory.
"""
import ctypes
import sys

# Load by absolute path rather than by soname. The shim is injected at a fixed
# location in the guest, and relying on the loader path picks up whatever the
# image happens to carry.
SHIM = "/opt/smolvm-cuda/libcuda.so.1"

try:
    lib = ctypes.CDLL(SHIM)
except OSError as exc:
    print(f"load_shim -> FAILED {exc}")
    print("If this says 'not found', the machine was started without --cuda.")
    print("If it names a musl or ld-linux problem, use a glibc image: the shim is glibc,")
    print("so an Alpine guest cannot load it. python:3.12-slim works and has Python already.")
    sys.exit(1)

print(f"load_shim -> ok {SHIM}")

rc = lib.cuInit(0)
print("cuInit ->", rc)

count = ctypes.c_int(-1)
print("cuDeviceGetCount ->", lib.cuDeviceGetCount(ctypes.byref(count)), "count =", count.value)

buf = ctypes.create_string_buffer(128)
print("cuDeviceGetName ->", lib.cuDeviceGetName(buf, 128, 0), "name =", buf.value.decode())

ctx = ctypes.c_void_p()
print("cuCtxCreate ->", lib.cuCtxCreate_v2(ctypes.byref(ctx), 0, 0))

free, total = ctypes.c_size_t(), ctypes.c_size_t()
print("cuMemGetInfo ->", lib.cuMemGetInfo_v2(ctypes.byref(free), ctypes.byref(total)),
      "total MiB =", total.value // (1024 * 1024))

# 1 MiB to the device and back; the first 16 bytes are compared.
N = 1024 * 1024
src = (ctypes.c_ubyte * N)(*([7] * 16 + [0] * (N - 16)))
dptr = ctypes.c_void_p()
print("cuMemAlloc   ->", lib.cuMemAlloc_v2(ctypes.byref(dptr), ctypes.c_size_t(N)))
print("cuMemcpyHtoD ->", lib.cuMemcpyHtoD_v2(dptr, src, ctypes.c_size_t(N)))
dst = (ctypes.c_ubyte * N)()
print("cuMemcpyDtoH ->", lib.cuMemcpyDtoH_v2(dst, dptr, ctypes.c_size_t(N)))
print("roundtrip first 16 bytes match:", list(dst[:16]) == [7] * 16)
print("cuMemFree    ->", lib.cuMemFree_v2(dptr))

scripts/cleanup.sh

#!/usr/bin/env bash
# Delete the machines this packet's scripts created, then prove the host is clean.
#
# Only machines recorded in the state file are deleted, with --cascade, so any
# machine branched from one of them goes too, whatever its name. Any other
# machine you or another session created by hand is never deleted. Scripts
# record a name by calling: cleanup.sh --record <name>
#
# usage: cleanup.sh [--record <name>] [--reap] [--purge]
#   --record <name>  add a machine name to the state file and exit
#   --reap           kill every VM process under this HOME; run without it first
#   --purge          also remove the state file once the list is empty

set -uo pipefail

PACKET="gpu-cuda"
PREFIX="smolskill-"

SMOLVM="${SMOLVM:-$(command -v smolvm 2>/dev/null)}"
STATE_DIR="${SMOLVM_SKILL_STATE_DIR:-${XDG_STATE_HOME:-$HOME/.local/state}/smolvm-skills}"
STATE_FILE="$STATE_DIR/$PACKET.machines"

reap=0
purge=0
while [ $# -gt 0 ]; do
    case "$1" in
        --record)
            mkdir -p "$STATE_DIR"
            printf '%s\n' "$2" >> "$STATE_FILE"
            exit 0
            ;;
        --reap)  reap=1 ;;
        --purge) purge=1 ;;
        *) printf 'unknown argument: %s\n' "$1" >&2; exit 2 ;;
    esac
    shift
done

if [ -z "$SMOLVM" ]; then
    printf 'smolvm not found; set SMOLVM to its path\n' >&2
    exit 2
fi

# With SMOLVM_DATA_DIR set, smolvm runs with HOME there on Linux, so its cache
# is under .cache in it.
case "$(uname -s)" in
    Darwin) VMS_DIR="$HOME/Library/Caches/smolvm/vms" ;;
    *)      cache_root="${SMOLVM_DATA_DIR:+$SMOLVM_DATA_DIR/.cache}"
            VMS_DIR="${cache_root:-${XDG_CACHE_HOME:-$HOME/.cache}}/smolvm/vms" ;;
esac
VMS_DIR="${SMOLVM_VMS_DIR:-$VMS_DIR}"
SMOLVM_PREFIX="${SMOLVM_PREFIX:-$HOME/.smolvm}"
if [ -n "${SMOLVM:-}" ]; then
    case "$(readlink "$SMOLVM" 2>/dev/null || printf '%s' "$SMOLVM")" in
        "$SMOLVM_PREFIX"/*) ;;
        *) printf 'note=%s is not under %s, so the process scan cannot see its forked VMs (all of its VMs on macOS); set SMOLVM_PREFIX to its directory\n' "$SMOLVM" "$SMOLVM_PREFIX" ;;
    esac
fi

# List this HOME's VM processes as "pid marker". The plain run path execs a
# `_boot-vm` child that carries its boot config; the pack-run path forks one that
# carries none. Linux names both `libkrun VM`, or `VM:<hostname>` when HOSTNAME
# is exported; on macOS the executable path and the parent chain scope the
# search. The teardown packet's traps have the why.
list_vm_processes() {
    case "$(uname -s)" in
        Linux)
            for p in /proc/[0-9]*; do
                case "$(cat "$p/comm" 2>/dev/null)" in "libkrun VM"|VM:*) ;; *) continue ;; esac
                pid="${p#/proc/}"
                cfg="$(tr '\0' '\n' < "$p/cmdline" 2>/dev/null | sed -n '3p')"
                case "$cfg" in
                    "$VMS_DIR"/*) printf '%s %s\n' "$pid" "$cfg"; continue ;;
                esac
                case "$(readlink "$p/exe" 2>/dev/null)" in
                    "$SMOLVM_PREFIX"/*) printf '%s forked-under %s\n' "$pid" "$SMOLVM_PREFIX" ;;
                esac
            done
            ;;
        Darwin)
            # shellcheck disable=SC2009  # pgrep -f would match this script.
            procs="$(ps -axo pid=,ppid=,command= 2>/dev/null)"
            printf '%s\n' "$procs" | while read -r pid ppid rest; do
                case "$rest" in "$SMOLVM_PREFIX"/smolvm-bin*) ;; *) continue ;; esac
                case "$rest" in
                    *" _boot-vm "*) printf '%s %s\n' "$pid" "${rest#* _boot-vm }"; continue ;;
                esac
                # An orphan counts only if it is a run; a fork has its parent's command line.
                if [ "$ppid" = 1 ]; then
                    case "$rest" in
                        *" machine run "*|*" vm run "*|*" pack run "*)
                            printf '%s orphaned-under %s\n' "$pid" "$SMOLVM_PREFIX" ;;
                    esac
                else
                    parent="$(printf '%s\n' "$procs" | while read -r q qp qrest; do [ "$q" = "$ppid" ] && { printf '%s' "$qrest"; break; }; done)"
                    [ "$parent" = "$rest" ] && printf '%s forked-under %s\n' "$pid" "$SMOLVM_PREFIX"
                fi
            done
            ;;
    esac
}

# 1. Delete recorded machines. Without --force a delete prompts and defaults to
# No; --cascade removes branch children, which otherwise block it.
if [ -s "$STATE_FILE" ]; then
    while read -r name; do
        [ -n "$name" ] || continue
        case "$name" in "$PREFIX"*) ;; *)
            printf 'skipping %s: not created by this packet (no %s prefix)\n' "$name" "$PREFIX"
            continue ;;
        esac
        # Only names still listed: a second stop of a missing name leaves a
        # directory that reads as a leak. The list is read first because grep -q
        # under pipefail can fail the pipeline and skip a listed machine.
        listed="$("$SMOLVM" machine list </dev/null 2>/dev/null | awk 'NR>2{print $1}')"
        grep -qx -- "$name" <<<"$listed" || continue
        "$SMOLVM" machine stop   --name "$name" </dev/null >/dev/null 2>&1
        "$SMOLVM" machine delete --name "$name" --force --cascade </dev/null 2>&1 | sed 's/^/  /'
    done < "$STATE_FILE"
fi

# 2. An ephemeral machine's entry retires after its run returns: poll, 20 s at most.
printf 'waiting=up to 20s for ephemeral entries to retire before asserting\n'
waited=0
listing="$("$SMOLVM" machine list 2>&1)"
while ! grep -q 'No machines found' <<<"$listing" && [ "$waited" -lt 20 ]; do
    sleep 1; waited=$((waited + 1))
    listing="$("$SMOLVM" machine list 2>&1)"
done
printf 'waited=%ss\n' "$waited"

# 3. Assert the value, not the exit code.
if grep -q 'No machines found' <<<"$listing"; then
    printf 'machines=clean\n'
    [ "$purge" -eq 1 ] && rm -f "$STATE_FILE"
else
    printf 'machines=remaining\n'
    printf '%s\n' "$listing" | sed 's/^/  /'
    printf 'note=this packet did not create these, so it will not delete them. By name:\n'
    printf '  smolvm machine stop --name <NAME> && smolvm machine delete --name <NAME> --force\n'
    printf '  add --cascade for a machine that was branched from another\n'
fi

# 4. Report VM processes under this HOME's state, such as a killed wrapper's.
# With --reap every one is killed, including a machine another packet kept.
found=0
while read -r pid cfg; do
    [ -n "$pid" ] || continue
    found=1
    printf 'vm_process=%s config=%s\n' "$pid" "$cfg"
    if [ "$reap" -eq 1 ]; then
        kill -9 "$pid" 2>/dev/null && printf '  killed %s\n' "$pid"
    fi
done <<EOF
$(list_vm_processes)
EOF

if [ "$found" -eq 0 ]; then
    printf 'vm_processes=none\n'
elif [ "$reap" -eq 0 ]; then
    printf 'rerun with --reap to kill them\n'
fi

CUDA traps

A zero exit code proves nothing on a remoted API

This is the trap the whole packet is shaped around. CUDA in smolvm is remoted, not passed through: the guest gets no NVIDIA driver and no /dev/nvidia*, and a compatibility libcuda.so.1 forwards driver calls over vsock to a host daemon that owns the device. A program can therefore start, link against libcuda.so.1, and exit zero without a GPU ever being reached.

Assert a device name and a computed or copied result. scripts/cuda-probe.py asserts both: cuDeviceGetName returning a real name, and a 1 MiB host to device to host round trip whose first 16 bytes come back as sent.

nvidia-smi is absent inside a --cuda guest, and that is correct

So is /dev/nvidia*. Both are the documented remoting design, not a fault, and neither is a useful check. Use the driver API.

Use a glibc image, and load the shim by absolute path

The injected shim is glibc, so an Alpine (musl) guest cannot load it. python:3.12-slim is small and already has Python.

Load it as /opt/smolvm-cuda/libcuda.so.1 rather than by soname: relying on the loader path picks up whatever the image happens to carry, and the shim is at a fixed location.

CUDA base images do not ship python3, so nvidia/cuda:...-base-... needs apt-get install -y python3 in the guest first, or a -devel image. That is a reason to prefer a small glibc image over a CUDA one for a probe.

A fresh GPU cloud instance does not have KVM access for your user

/dev/kvm is crw-rw---- root:kvm and your account is not in the kvm group. The installer warns and continues, so the install succeeds and the first VM fails. Apply the group without logging out:

sudo usermod -aG kvm "$USER"
sg kvm -c 'smolvm machine run --mem 2048 --net --image alpine -- echo OK'

“agent did not become ready within 30 seconds” with --cuda was the guest’s size, not the GPU

Observed on Ubuntu 24.04 aarch64 on 2026-09-07, running machine run --cuda on a host with no NVIDIA hardware:

Starting ephemeral machine (vm-5d98e4be)...
Error: agent operation failed: start machine: agent operation failed: wait for ready:
agent did not become ready within 30 seconds

Re-measured on v1.18.2, 2026-09-24, and the missing GPU was not the cause. On the same kind of host, Lima linux-kvm with no NVIDIA hardware, scripts/run-cuda-probe.sh at its --mem 8192 failed with the message above; the same probe at --mem 1024 reached smolvm CPU emulation device and returned result=cpu_emulation_not_gpu; and machine run --mem 8192 --net --image alpine with no --cuda failed with the same message. That host booted 2048 MiB and not 2560 on 2026-09-24. So the timeout is the guest’s size on that host, and a host that cannot load the driver library answers --cuda with the emulation device instead.

Large CUDA images may not pull

nvidia/cuda:12.4.1-base-ubuntu22.04 failed after 582 s with crane blob failed ... unexpected EOF on the Windows host’s network. That is a transfer failure, not a CUDA one, but it is another reason to prefer a small image.

Memory: give the machine real headroom

The Linux GPU runs used --mem 8192, and the Windows probe line in references/windows.md uses --mem 6144. On a small host, see the ready-timeout trap in the install packet: the 30 s limit is a hard-coded constant and no flag raises it for machine run or machine start.

--cuda changes nothing on the host

The remoting shim is injected at /opt/smolvm-cuda inside the guest only, and no host NVIDIA state is touched. Verified after a series of CUDA runs plus a Kubernetes install and teardown on the same host: nvidia-smi still reported the device, and a whole-filesystem sweep for *smolvm* came back empty.

--cuda and --gpu are different features

--cuda is compute over vsock. --gpu is Vulkan graphics over virtio-gpu; on the hosts tested through v1.16.1 the host renderer reported the Venus capset at version 0, and on v1.18.2 on macOS arm64 a guest reached Virtio-GPU Venus (Apple M4). docs/gpu.md in the repository covers it. They share nothing but the word GPU. On Windows, on v1.14.6 and v1.22.2, --gpu boots and the guest has no GPU device; nothing reports that the flag was dropped.