Docker in machine: run a Docker daemon inside a machine
Runs a Docker daemon inside a smolvm machine and shows it working there, which is what tools that call Docker themselves need, such as a test suite that starts containers, an image build, or a coding agent that launches containers. Use when dockerd will not start inside a machine; when Docker worked on the first boot and broke after a stop and start; when deciding where Docker’s data directory has to live; or when checking whether this is possible on a given platform at all. Do not use it to run OCI images, which smolvm boots natively without Docker, and do not attempt it on Windows, where the bundled guest kernel cannot support it.
Verified on smolvm v1.34.0 on macOS arm64 and Linux x86_64, 2026-11-09, and on v1.18.2 on Linux aarch64, 2026-09-24; the aarch64 runs used
the Smolfile with memory = 1024, for a host reason given below. Done means docker info succeeds inside the guest, a nested container runs, and
Docker’s data sits on the ext4 storage disk rather than the rootfs overlay.
The Linux aarch64 runs used the scripts of their date; this version’s preflight and cleanup scripts ran
on Linux aarch64 on v1.22.2 on 2026-10-03.
On Windows, up to and including v1.22.2, dockerd does not start in a machine. The Windows
guest kernel is built without bridge networking and POSIX message queues, so dockerd cannot
create its default network and, forced past that, containers still cannot be created. references/windows.md has the evidence and the direct kernel probe, which gave the same answer on
v1.22.2.
Docker’s data has to be on the machine’s disk on every start
From v1.28.1 a disk volume, a guest path with no host side, is a directory on the machine’s ext4
disk, bound into the workload on every start (#1787). assets/docker.smolfile declares /var/lib/docker and /var/lib/containerd that way, and its docker:dind image starts dockerd itself on every start, so nothing has to be re-applied after a
stop and start.
On v1.28.0 and earlier, init runs once, so the bind mounts are gone on the second boot and
have to be re-applied on every start. assets/docker-bind-mounts.smolfile is that configuration,
and scripts/start-dockerd.sh re-applies the mounts and starts the daemon, as the upstream
example’s “Start dockerd” recipe does. Reproduced on both hosts on v1.14.2, v1.18.2 and v1.22.2,
on macOS on v1.16.1, and with that file on v1.29.0:
Init already completed, skipping 5 command(s)
NO_BIND_MOUNTS_AFTER_RESTART
DOCKERD_DOWN Running scripts/start-dockerd.sh afterwards restored everything, and docker images still
listed alpine:latest, because the images are on /storage. The failure mode is a daemon that
will not start, or one running on the wrong filesystem, not lost data.
init runs on the first start only, as smolfile.md says, whatever machine create --help said before v1.23.7. More in references/traps.md.
Procedure
1. Preflight.
scripts/preflight.sh docker_in_machine=verified on macOS and Linux, unavailable on Windows with the reason.
2. Create. assets/docker.smolfile, for v1.28.1 and later, is docker:dind with the two disk
volumes. On v1.28.0 and earlier pass assets/docker-bind-mounts.smolfile, the upstream example’s
configuration, shipped here because the release tarball does not contain examples/.
scripts/create-docker-machine.sh # name defaults to smolskill-docker
scripts/create-docker-machine.sh smolskill-docker assets/docker-bind-mounts.smolfile # v1.28.0 and earlier With the bind-mount file the apk add docker happens on the first start and dominates the
time.
3. Wait for the daemon. Run this on every start, not only the first.
scripts/start-dockerd.sh With the disk volumes it waits for the image’s own dockerd to answer docker info. With the
bind-mount file, where nothing starts the daemon, it re-applies both bind mounts, clears a stale
pid file and starts dockerd with the overlay2 driver, printing starting dockerd on the bind mounts. On v1.29.0 with the volumes:
Server Version: 29.9.0
Storage Driver: overlayfs
Docker Root Dir: /var/lib/docker
server_version=29.9.0
result=dockerd_up 4. Prove it, on the right filesystem.
scripts/verify-docker.sh storage_driver=ok (overlayfs)
docker_root_device=ok (/dev/vda)
pull=docker.io/library/alpine:latest
nested_container=ok (NESTED_OK)
host_network=ok (HOSTNET_OK)
host_socket=present (.../vms/<hash>/docker.sock)
result=docker_ok overlayfs is the name Docker 29’s containerd image store reports; Docker 25 on the bind-mount
file reports overlay2, and the check takes either.
docker_root_device is the check that matters. docker info succeeds while /var/lib/docker sits on the rootfs overlay, and the failure that follows is confusing and much later.
5. Use it, and clean up only when it is no longer wanted. A machine someone asked for is the
deliverable: leave it running and give them its name. Their own code gets in with smolvm machine cp <file> smolskill-docker:/workspace/<file> or a -v mount on create, and runs
with smolvm machine exec --name smolskill-docker -- ...; with no docker client on the host, code
that calls Docker runs inside the machine against this daemon. What this packet shows is the
daemon: docker info, a nested container and host networking.
scripts/cleanup.sh --purge cleanup.sh waits up to 20 seconds, polling the machine list, and prints waiting=up to 20s first: an ephemeral machine’s entry retires after its run returns.
Why Docker’s data has to live on the machine’s disk
A hard requirement, not a preference. The smolvm rootfs overlay uses the initramfs (ramfs) as its
lower layer, ramfs has no file-handle support, and overlayfs then rejects it as an upper dir for
Docker’s nested overlay. Both mounts are needed: /var/lib/containerd holds the snapshotter’s
overlay state and fails the same way.
That is why both Smolfiles declare storage = 20 and put the two directories on it, as disk
volumes or as bind mounts from /storage, and why every check here is against /dev/vda.
The host-side socket
docker_socket = true bridges the guest’s /var/run/docker.sock to a host path under the
machine’s data directory:
D=$(smolvm machine data-dir --name smolskill-docker)
DOCKER_HOST=unix://$D/docker.sock docker ps # needs a docker client on the host The socket is created and verify-docker.sh asserts it exists. Driving it from the host was not
verified: neither host used here has a docker client.
Security defaults, and why they are the defaults
docker_socket = trueis the one line here that gives something outside the VM real authority. A process on the host that can open that socket can start containers inside the machine, mount paths the machine can see and read anything they hold. Leave it off unless a host tool actually needs it, which is why nothing in this packet’s own checks depends on it.- A Docker daemon inside the machine is root inside the machine, and that is the point. The VM boundary is what makes it acceptable: treat root in the guest as untrusted, as the security model says, and every forwarded mount, port and network permission becomes part of what the nested containers can reach.
--network=hostinside the guest is the guest’s network, not yours. It is verified here because Testcontainers and Compose commonly need it, and it is contained by the VM rather than by Docker.net = trueis outbound access for the whole machine, and it is required for this use case becauseapk add dockerand everydocker pullneed it. Scope it with--allow-hostwhere the set of registries is known.- The scripts wrap the public CLI only. They edit no smolvm configuration and nothing under
~/.smolvm, and cleanup deletes only names it recorded under thesmolskill-prefix.
Platform arms
- macOS arm64: v1.34.0 and v1.29.0 with the disk volumes, every check passed before and after a stop and start; the bind-mount file on v1.29.0 too, the restart trap and its recovery included.
- Linux aarch64: v1.18.2 and once on v1.22.2, with the Smolfile’s memory at 1024 MiB: the Lima
host could not boot larger guests inside the fixed 30 s readiness window, and the
installpacket’s traps have the numbers. On a small or busy host, lowermemorybefore suspecting Docker; 1024 MiB was enough fordockerdand a nested alpine container. - Linux x86_64: verified on v1.34.0, twice, on an NVIDIA A10 cloud host, with the Smolfile as shipped.
- Windows x86_64: does not run on v1.22.2. A v1.14.2 run showed
dockerdfailing; on 2026-10-03 on v1.22.2 (Windows 11 Home build 10.0.26200 UBR 9457) the direct kernel probe in a plainalpineguest gaveRTNETLINK answers: Not supportedfor a bridge andNo such deviceformqueue.references/windows.mdhas both.
Eval prompts, and what they produced
1. “Give me a machine with a Docker daemon inside it, and show me Docker actually works in it.”
On v1.18.2, both hosts identical:
docker_version=Docker version 25.0.5, build d260a54c81efcc3f00fe67dee78c94b16c2f8692
result=installed
server_version=25.0.5
result=dockerd_up
storage_driver=ok (overlay2)
docker_root_device=ok (/dev/vda)
nested_container=ok (NESTED_OK)
host_network=ok (HOSTNET_OK)
result=docker_ok 2. “Docker worked yesterday and today dockerd will not start. Nothing changed.”
Reproduced on both hosts by stopping and starting the machine, with the lines shown under the
trap above; scripts/start-dockerd.sh then brought it back to result=docker_ok with alpine:latest still listed.
3. “Can I do this on Windows?”
No, and the answer is short enough to give without running anything: the bundled guest kernel
has CONFIG_BRIDGE and CONFIG_POSIX_MQUEUE both off, so dockerd fails on error creating default "bridge" network: operation not supported, and with --bridge=none the
daemon comes up healthy while every container fails on /dev/mqueue. That result is from an
earlier run on Windows 11 against v1.14.2, and the kernel probe on references/windows.md gave
the same two answers on v1.22.2.
Re-verified on v1.34.0
Run 2026-11-09 PT against v1.34.0 from the published release, checksum checked, under a fresh
isolated HOME on macOS 27.2 arm64 (Apple M4), twice, the second time from a fresh HOME, and once
on Lima linux-kvm (Ubuntu 24.04 aarch64, 11 GiB). Every step on both hosts with the disk volumes:
Docker 29.9.3, docker_root_device=ok (/dev/vda), nested_container=ok and result=docker_ok,
again after a stop and start.
On Linux x86_64 (Ubuntu 22.04, KVM, an NVIDIA A10 cloud host with 30 cores and 222 GiB), twice on
2026-11-09, each pass from a fresh HOME, every step with the disk volumes: Docker 29.9.3, docker_root_device=ok (/dev/vda), nested_container=ok (NESTED_OK) and result=docker_ok, again
after a stop and start.
Re-verified on v1.29.0, with the disk volumes
Run 2026-10-15 PT against v1.29.0 from the published release (the darwin-arm64 archive, checksum
checked), under a fresh isolated HOME on macOS 27.0.1 arm64, twice, the second time from a fresh HOME. Linux was not run.
create-docker-machine.sh gave result=installed (Docker 29.9.0) in 17 s, start-dockerd.sh result=dockerd_up without starting a daemon of its own, and verify-docker.sh storage_driver=ok (overlayfs), docker_root_device=ok (/dev/vda), nested_container=ok (NESTED_OK), host_network=ok (HOSTNET_OK), host_socket=present and result=docker_ok. After a
stop and start df put both directories on /dev/vda, alpine:latest was still listed, and the
same checks passed. The first run’s verify-docker.sh expected overlay2 and failed on that line
alone; the check now takes either name. With assets/docker-bind-mounts.smolfile, start-dockerd.sh printed starting dockerd on the bind mounts after each start and verify-docker.sh gave result=docker_ok.
Re-verified on v1.29.0, with the bind mounts
Run 2026-10-15 PT against v1.29.0 from the published release, checksum checked, under a fresh
isolated HOME on macOS 27.0.1 arm64, the scripts twice, the second time from a fresh HOME, and
the checks beyond them once. Linux was not run, so the Linux stamp stays on its earlier release.
create-docker-machine.sh gave result=installed (Docker 25.0.5), start-dockerd.sh result=dockerd_up, and verify-docker.sh docker_root_device=ok (/dev/vda), nested_container=ok (NESTED_OK) and result=docker_ok. After a stop and start the machine
printed Init already completed, skipping 5 command(s), NO_BIND_MOUNTS_AFTER_RESTART and DOCKERD_DOWN; start-dockerd.sh brought it back and alpine:latest was still listed.
Re-verified on v1.22.2
Run 2026-10-03 PT against v1.22.2 from the published release, checksum checked, under an isolated HOME on macOS 27.0.1 arm64, twice, the second time from a fresh HOME. On Lima linux-kvm (Ubuntu 24.04 aarch64) guests above 2048 MiB timed out on 2026-10-03, so the Linux lines below are
a single run and the Linux stamp stays on its earlier release.
macOS: result=docker_ok with docker_root_device=ok (/dev/vda), then after a stop and start Init already completed, skipping 5 command(s), NO_BIND_MOUNTS_AFTER_RESTART, DOCKERD_DOWN,
and start-dockerd.sh brought it back with alpine:latest still listed.
Linux aarch64 at memory = 1024: the same lines.
What was not run
dockerditself on Windows after v1.14.2. The v1.22.2 answer is the kernel probe, not a daemon run.- Driving the host-side
docker.sockfrom a hostdockerclient. The socket is created and asserted; neither host here has a docker client to drive it with. - Compose and Testcontainers themselves. The packet verifies
docker info, a nested container and host networking, which is what those depend on, not the tools.
Related packets
dev-envfor theinit-runs-once semantics this is built on.installfor the boot this assumes,teardownfor the cleanup script.
Scripts
The files the procedure runs, in the order it runs them. It calls each one by the path in its heading, relative to the directory the procedure is saved in.
scripts/preflight.sh
#!/usr/bin/env bash
# Report whether this host can run a Docker daemon inside a smolvm machine.
# Read-only: starts no VM, writes no smolvm state.
#
# Output is one key=value per line so a caller can parse it. The last line is
# always result=ready or result=blocked.
set -uo pipefail
VERIFIED_VERSION="1.34.0"
emit() { printf '%s=%s\n' "$1" "$2"; }
blocked=0
note() { printf 'note=%s\n' "$1"; }
# --- the binary --------------------------------------------------------------
SMOLVM="${SMOLVM:-$(command -v smolvm 2>/dev/null)}"
if [ -z "$SMOLVM" ]; then
emit smolvm_installed no
emit smolvm_version ""
blocked=1
else
emit smolvm_installed yes
version="$("$SMOLVM" --version 2>/dev/null | awk '{print $NF}')"
emit smolvm_version "${version:-unknown}"
fi
emit verified_version "$VERIFIED_VERSION"
if [ -n "${version:-}" ] && [ "$version" != "unknown" ]; then
if [ "$version" = "$VERIFIED_VERSION" ]; then
emit version_status match
else
newest="$(printf '%s\n%s\n' "$version" "$VERIFIED_VERSION" | sort -V | tail -1)"
if [ "$newest" = "$version" ]; then
emit version_status newer
note "this packet was verified on $VERIFIED_VERSION and the binary is $version; flags and messages move every release, so check the output against the binary before trusting a step here"
else
emit version_status older
note "this packet was verified on $VERIFIED_VERSION and the binary is $version"
fi
fi
else
emit version_status unknown
fi
# --- platform ----------------------------------------------------------------
kernel="$(uname -s)"
arch="$(uname -m)"
case "$arch" in aarch64|arm64) arch=aarch64 ;; esac
case "$kernel" in
Darwin)
emit platform "darwin-$arch"
emit accel hvf
emit macos_version "$(sw_vers -productVersion)"
hv="$(sysctl -n kern.hv_support 2>/dev/null)"
if [ "$hv" = "1" ]; then emit accel_access ok; else emit accel_access denied; blocked=1; fi
if [ "$arch" != "aarch64" ]; then
emit hardware_verified no
note "Intel Mac is not verified by this packet; the installer accepts it and nothing here was run on one"
else
emit hardware_verified yes
fi
# A VM's agent socket lives under the cache directory. macOS sockaddr_un
# holds 104 bytes including the terminator.
sock="$HOME/Library/Caches/smolvm/vms/0123456789abcdef/agent.sock"
len=${#sock}
emit socket_path_bytes "$len"
if [ "$len" -gt 100 ]; then
emit socket_path_status too_long
blocked=1
note "HOME is too deep: every VM start will fail. From v1.28.0 the error names the socket path and its length; before it, krun_start_enter -22, whose text blames disks and device options. Install under a shorter HOME."
else
emit socket_path_status ok
fi
emit unsupported "cuda"
emit docker_in_machine verified
;;
Linux)
emit platform "linux-$arch"
emit accel kvm
emit socket_path_status n_a
if [ ! -e /dev/kvm ]; then
emit accel_access missing
blocked=1
note "/dev/kvm does not exist; this host has no KVM"
elif [ -r /dev/kvm ] && [ -w /dev/kvm ]; then
emit accel_access ok
else
emit accel_access denied
blocked=1
note "your user cannot open /dev/kvm. The installer warns and continues, so a successful install says nothing about whether a VM will start. Fix: sudo usermod -aG kvm \$USER, then run the next command through sg kvm -c '...' rather than logging out."
fi
emit docker_in_machine verified
;;
*)
emit platform "unsupported-$kernel"
emit accel unknown
emit accel_access unknown
blocked=1
emit docker_in_machine unavailable
note "Docker in a machine cannot work on Windows: the bundled guest kernel has neither bridge networking nor POSIX message queues, so dockerd will not start and, forced past that, containers still fail. See references/windows.md."
;;
esac
if [ "$blocked" -eq 0 ]; then emit result ready; else emit result blocked; fi assets/docker.smolfile
# A machine that runs a Docker daemon inside itself, for workloads that must
# call Docker themselves: Testcontainers, Compose, image builds, agents that
# launch containers. smolvm boots OCI images without Docker, so this is only for
# software inside the machine that needs the daemon.
#
# From v1.28.1. The docker:dind image starts dockerd as its workload on every
# start, and the two volumes are directories on the machine's ext4 disk, bound
# into the workload on every start. On v1.28.0 and earlier, which have no disk
# volumes, use docker-bind-mounts.smolfile.
image = "docker:dind"
cpus = 2
memory = 2048
net = true
# Docker's data directory must live on the machine's disk, not on the rootfs
# overlay: the overlay's lower layer is ramfs, which has no file-handle support,
# and overlayfs then rejects it as an upper dir for Docker's nested overlay.
storage = 20
volumes = ["/var/lib/docker", "/var/lib/containerd"]
docker_socket = true assets/docker-bind-mounts.smolfile
# A machine that runs a Docker daemon inside itself, for workloads that must
# call Docker themselves: Testcontainers, Compose, image builds, agents that
# launch containers. smolvm boots OCI images without Docker, so this is only for
# software inside the machine that needs the daemon.
#
# For v1.28.0 and earlier, which have no disk volumes; from v1.28.1 use
# docker.smolfile. The bind mounts below are in `init`, as in the upstream
# example, and `init` runs ONCE: they hold for the first boot only.
# scripts/start-dockerd.sh, the example's "Start dockerd" recipe, re-applies
# them on every start.
cpus = 2
memory = 2048
net = true
# Docker's data directory must live on /storage, not on the rootfs overlay.
# This is a hard requirement: the rootfs overlay uses the initramfs (ramfs) as
# its lower layer, ramfs has no file-handle support, and overlayfs then rejects
# it as an upper dir for Docker's nested overlay.
storage = 20
docker_socket = true
init = [
"apk update -q",
"apk add docker -q",
"mkdir -p /storage/docker /var/lib/docker /storage/containerd /var/lib/containerd",
"mount --bind /storage/docker /var/lib/docker",
"mount --bind /storage/containerd /var/lib/containerd",
] scripts/create-docker-machine.sh
#!/usr/bin/env bash
# Create the machine and install Docker into it. The install happens on the
# first start, not on create.
#
# usage: create-docker-machine.sh [<name>] [<smolfile>]
set -uo pipefail
here="$(cd "$(dirname "$0")" && pwd)"
NAME="${1:-smolskill-docker}"
SMOLFILE="${2:-$here/../assets/docker.smolfile}"
SMOLVM="${SMOLVM:-$(command -v smolvm 2>/dev/null)}"
if [ -z "$SMOLVM" ]; then
printf 'smolvm not found; set SMOLVM to its path\n' >&2
exit 2
fi
case "$NAME" in
smolskill-*) ;;
*) printf 'name must start with smolskill- so cleanup.sh will delete it\n' >&2; exit 2 ;;
esac
"$SMOLVM" machine create --name "$NAME" -s "$SMOLFILE" --net-backend virtio-net 2>&1 | sed 's/^/ /'
"$here/cleanup.sh" --record "$NAME"
start_out="$("$SMOLVM" machine start --name "$NAME" 2>&1)"
printf '%s\n' "$start_out" | sed 's/^/ /'
fail=0
if printf '%s' "$start_out" | grep -q "Machine '$NAME' running"; then
printf 'machine_running=yes\n'
else
printf 'machine_running=no\n'
fail=1
fi
docker_version="$("$SMOLVM" machine exec --name "$NAME" -- docker --version 2>&1 | tr -d '\r')"
printf 'docker_version=%s\n' "$docker_version"
case "$docker_version" in
"Docker version"*) ;;
*) fail=1 ;;
esac
if [ "$fail" -eq 0 ]; then printf 'result=installed\n'; else printf 'result=failed\n'; fi
exit "$fail" scripts/start-dockerd.sh
#!/usr/bin/env bash
# Wait for dockerd inside the machine, starting it when nothing else does.
#
# usage: start-dockerd.sh [<name>]
#
# Run this on EVERY start, not only the first. With assets/docker.smolfile
# (v1.28.1 and later) the docker:dind workload starts dockerd on its disk
# volumes, and this only waits for it. With assets/docker-bind-mounts.smolfile
# nothing starts it, and `init` runs once, so on any start after the first
# /var/lib/docker is back on the rootfs overlay: this re-applies the bind mounts
# and starts dockerd, the upstream example's "Start dockerd" recipe.
set -uo pipefail
NAME="${1:-smolskill-docker}"
SMOLVM="${SMOLVM:-$(command -v smolvm 2>/dev/null)}"
if [ -z "$SMOLVM" ]; then
printf 'smolvm not found; set SMOLVM to its path\n' >&2
exit 2
fi
# shellcheck disable=SC2016 # $(seq ...) runs in the guest shell, not here
"$SMOLVM" machine exec --name "$NAME" -- sh -c '
# 10 s for the image'\''s own dockerd to appear before deciding nothing starts it.
for i in $(seq 1 10); do pgrep -x dockerd >/dev/null && break; sleep 1; done
if ! pgrep -x dockerd >/dev/null; then
echo "starting dockerd on the bind mounts"
mkdir -p /storage/docker /var/lib/docker /storage/containerd /var/lib/containerd
mountpoint -q /var/lib/docker || mount --bind /storage/docker /var/lib/docker
mountpoint -q /var/lib/containerd || mount --bind /storage/containerd /var/lib/containerd
rm -f /var/run/docker.pid
dockerd --storage-driver=overlay2 >/tmp/dockerd.log 2>&1 &
fi
# 60 s: dockerd answered docker info within a few seconds on both hosts here;
# the margin covers a first start that has to create its storage layout.
for i in $(seq 1 60); do docker info >/dev/null 2>&1 && break; sleep 1; done
docker info 2>/dev/null | grep -E "Server Version|Storage Driver|Docker Root Dir"
' 2>&1 | sed 's/^/ /'
# `docker info` succeeding is not the check that matters; see verify-docker.sh.
server="$("$SMOLVM" machine exec --name "$NAME" -- docker info --format '{{.ServerVersion}}' 2>&1 | tr -d '\r')"
printf 'server_version=%s\n' "$server"
case "$server" in
[0-9]*) printf 'result=dockerd_up\n' ;;
*)
printf 'result=dockerd_down\n'
printf 'daemon log:\n'
"$SMOLVM" machine exec --name "$NAME" -- tail -20 /tmp/dockerd.log 2>&1 | sed 's/^/ /'
exit 1
;;
esac scripts/verify-docker.sh
#!/usr/bin/env bash
# Prove Docker is usable AND on the right filesystem.
#
# usage: verify-docker.sh [<name>]
#
# The df assertion is the one that matters. `docker info` succeeds while
# /var/lib/docker sits on the rootfs overlay, and the failure that follows is
# confusing and much later: overlayfs rejects the ramfs-backed rootfs as an
# upper dir for Docker's nested overlay.
set -uo pipefail
NAME="${1:-smolskill-docker}"
SMOLVM="${SMOLVM:-$(command -v smolvm 2>/dev/null)}"
if [ -z "$SMOLVM" ]; then
printf 'smolvm not found; set SMOLVM to its path\n' >&2
exit 2
fi
fail=0
check() {
if [ "$2" = "$3" ]; then
printf '%s=ok (%s)\n' "$1" "$2"
else
printf '%s=FAIL expected=%s actual=%s\n' "$1" "$3" "$2"
fail=1
fi
}
exec_in() { "$SMOLVM" machine exec --name "$NAME" -- "$@" 2>&1 | tr -d '\r'; }
# An overlay driver: overlay2, or overlayfs, the name Docker 29's containerd
# image store reports (docker:dind on v1.29.0).
driver="$(exec_in docker info --format '{{.Driver}}')"
case "$driver" in overlayfs) check storage_driver "$driver" overlayfs ;; *) check storage_driver "$driver" overlay2 ;; esac
# The backing device, not the path. Both readings print /var/lib/docker.
# shellcheck disable=SC2016 # the awk field reference belongs to the guest shell
check docker_root_device "$(exec_in sh -c 'df /var/lib/docker | tail -1 | awk "{print \$1}"')" /dev/vda
# Pull first, and separately. A `docker run` that pulls interleaves the pull's
# progress with the container's output, and the two streams arrive in a
# different order on different hosts, so the marker is not reliably the last
# line. Pulling first leaves the run's output alone without discarding the
# stderr that would explain a real failure.
printf 'pull=%s\n' "$(exec_in docker pull -q alpine | tail -1)"
check nested_container "$(exec_in docker run --rm alpine echo NESTED_OK)" NESTED_OK
check host_network "$(exec_in docker run --rm --network=host alpine echo HOSTNET_OK)" HOSTNET_OK
# docker_socket = true bridges the guest's /var/run/docker.sock to a host path.
sock="$("$SMOLVM" machine data-dir --name "$NAME" 2>/dev/null)/docker.sock"
if [ -S "$sock" ]; then
printf 'host_socket=present (%s)\n' "$sock"
else
printf 'host_socket=absent (%s)\n' "$sock"
fi
if [ "$fail" -eq 0 ]; then printf 'result=docker_ok\n'; else printf 'result=FAILED\n'; fi
exit "$fail" scripts/cleanup.sh
#!/usr/bin/env bash
# Delete the machines this packet's scripts created, then prove the host is clean.
#
# Only machines recorded in the state file are deleted, with --cascade, so any
# machine branched from one of them goes too, whatever its name. Any other
# machine you or another session created by hand is never deleted. Scripts
# record a name by calling: cleanup.sh --record <name>
#
# usage: cleanup.sh [--record <name>] [--reap] [--purge]
# --record <name> add a machine name to the state file and exit
# --reap kill every VM process under this HOME; run without it first
# --purge also remove the state file once the list is empty
set -uo pipefail
PACKET="docker-in-machine"
PREFIX="smolskill-"
SMOLVM="${SMOLVM:-$(command -v smolvm 2>/dev/null)}"
STATE_DIR="${SMOLVM_SKILL_STATE_DIR:-${XDG_STATE_HOME:-$HOME/.local/state}/smolvm-skills}"
STATE_FILE="$STATE_DIR/$PACKET.machines"
reap=0
purge=0
while [ $# -gt 0 ]; do
case "$1" in
--record)
mkdir -p "$STATE_DIR"
printf '%s\n' "$2" >> "$STATE_FILE"
exit 0
;;
--reap) reap=1 ;;
--purge) purge=1 ;;
*) printf 'unknown argument: %s\n' "$1" >&2; exit 2 ;;
esac
shift
done
if [ -z "$SMOLVM" ]; then
printf 'smolvm not found; set SMOLVM to its path\n' >&2
exit 2
fi
# With SMOLVM_DATA_DIR set, smolvm runs with HOME there on Linux, so its cache
# is under .cache in it.
case "$(uname -s)" in
Darwin) VMS_DIR="$HOME/Library/Caches/smolvm/vms" ;;
*) cache_root="${SMOLVM_DATA_DIR:+$SMOLVM_DATA_DIR/.cache}"
VMS_DIR="${cache_root:-${XDG_CACHE_HOME:-$HOME/.cache}}/smolvm/vms" ;;
esac
VMS_DIR="${SMOLVM_VMS_DIR:-$VMS_DIR}"
SMOLVM_PREFIX="${SMOLVM_PREFIX:-$HOME/.smolvm}"
if [ -n "${SMOLVM:-}" ]; then
case "$(readlink "$SMOLVM" 2>/dev/null || printf '%s' "$SMOLVM")" in
"$SMOLVM_PREFIX"/*) ;;
*) printf 'note=%s is not under %s, so the process scan cannot see its forked VMs (all of its VMs on macOS); set SMOLVM_PREFIX to its directory\n' "$SMOLVM" "$SMOLVM_PREFIX" ;;
esac
fi
# List this HOME's VM processes as "pid marker". The plain run path execs a
# `_boot-vm` child that carries its boot config; the pack-run path forks one that
# carries none. Linux names both `libkrun VM`, or `VM:<hostname>` when HOSTNAME
# is exported; on macOS the executable path and the parent chain scope the
# search. The teardown packet's traps have the why.
list_vm_processes() {
case "$(uname -s)" in
Linux)
for p in /proc/[0-9]*; do
case "$(cat "$p/comm" 2>/dev/null)" in "libkrun VM"|VM:*) ;; *) continue ;; esac
pid="${p#/proc/}"
cfg="$(tr '\0' '\n' < "$p/cmdline" 2>/dev/null | sed -n '3p')"
case "$cfg" in
"$VMS_DIR"/*) printf '%s %s\n' "$pid" "$cfg"; continue ;;
esac
case "$(readlink "$p/exe" 2>/dev/null)" in
"$SMOLVM_PREFIX"/*) printf '%s forked-under %s\n' "$pid" "$SMOLVM_PREFIX" ;;
esac
done
;;
Darwin)
# shellcheck disable=SC2009 # pgrep -f would match this script.
procs="$(ps -axo pid=,ppid=,command= 2>/dev/null)"
printf '%s\n' "$procs" | while read -r pid ppid rest; do
case "$rest" in "$SMOLVM_PREFIX"/smolvm-bin*) ;; *) continue ;; esac
case "$rest" in
*" _boot-vm "*) printf '%s %s\n' "$pid" "${rest#* _boot-vm }"; continue ;;
esac
# An orphan counts only if it is a run; a fork has its parent's command line.
if [ "$ppid" = 1 ]; then
case "$rest" in
*" machine run "*|*" vm run "*|*" pack run "*)
printf '%s orphaned-under %s\n' "$pid" "$SMOLVM_PREFIX" ;;
esac
else
parent="$(printf '%s\n' "$procs" | while read -r q qp qrest; do [ "$q" = "$ppid" ] && { printf '%s' "$qrest"; break; }; done)"
[ "$parent" = "$rest" ] && printf '%s forked-under %s\n' "$pid" "$SMOLVM_PREFIX"
fi
done
;;
esac
}
# 1. Delete recorded machines. Without --force a delete prompts and defaults to
# No; --cascade removes branch children, which otherwise block it.
if [ -s "$STATE_FILE" ]; then
while read -r name; do
[ -n "$name" ] || continue
case "$name" in "$PREFIX"*) ;; *)
printf 'skipping %s: not created by this packet (no %s prefix)\n' "$name" "$PREFIX"
continue ;;
esac
# Only names still listed: a second stop of a missing name leaves a
# directory that reads as a leak. The list is read first because grep -q
# under pipefail can fail the pipeline and skip a listed machine.
listed="$("$SMOLVM" machine list </dev/null 2>/dev/null | awk 'NR>2{print $1}')"
grep -qx -- "$name" <<<"$listed" || continue
"$SMOLVM" machine stop --name "$name" </dev/null >/dev/null 2>&1
"$SMOLVM" machine delete --name "$name" --force --cascade </dev/null 2>&1 | sed 's/^/ /'
done < "$STATE_FILE"
fi
# 2. An ephemeral machine's entry retires after its run returns: poll, 20 s at most.
printf 'waiting=up to 20s for ephemeral entries to retire before asserting\n'
waited=0
listing="$("$SMOLVM" machine list 2>&1)"
while ! grep -q 'No machines found' <<<"$listing" && [ "$waited" -lt 20 ]; do
sleep 1; waited=$((waited + 1))
listing="$("$SMOLVM" machine list 2>&1)"
done
printf 'waited=%ss\n' "$waited"
# 3. Assert the value, not the exit code.
if grep -q 'No machines found' <<<"$listing"; then
printf 'machines=clean\n'
[ "$purge" -eq 1 ] && rm -f "$STATE_FILE"
else
printf 'machines=remaining\n'
printf '%s\n' "$listing" | sed 's/^/ /'
printf 'note=this packet did not create these, so it will not delete them. By name:\n'
printf ' smolvm machine stop --name <NAME> && smolvm machine delete --name <NAME> --force\n'
printf ' add --cascade for a machine that was branched from another\n'
fi
# 4. Report VM processes under this HOME's state, such as a killed wrapper's.
# With --reap every one is killed, including a machine another packet kept.
found=0
while read -r pid cfg; do
[ -n "$pid" ] || continue
found=1
printf 'vm_process=%s config=%s\n' "$pid" "$cfg"
if [ "$reap" -eq 1 ]; then
kill -9 "$pid" 2>/dev/null && printf ' killed %s\n' "$pid"
fi
done <<EOF
$(list_vm_processes)
EOF
if [ "$found" -eq 0 ]; then
printf 'vm_processes=none\n'
elif [ "$reap" -eq 0 ]; then
printf 'rerun with --reap to kill them\n'
fi Docker in a machine traps
The bind mounts do not survive a stop, and init will not re-apply them
This is the trap that breaks the second session, and it is the reason this packet has a
separate start-dockerd.sh rather than a Smolfile alone.
init runs once. On every start after the first the guest comes up with /var/lib/docker back on the rootfs overlay. Observed on macOS arm64 and Linux aarch64, immediately after one stop
and start:
Init already completed, skipping 5 command(s)
NO_BIND_MOUNTS_AFTER_RESTART
DOCKERD_DOWN Mounts applied in init hold for the first boot only. Re-apply them in the same command that
starts dockerd, as the upstream example’s “Start dockerd” recipe does and scripts/start-dockerd.sh does. After it ran, the same host reported dockerd_up and every check passed again.
Your images do survive, because they are on /storage. After the restart and the re-mount, docker images still listed alpine:latest. So the failure mode is “dockerd will not start” or
“dockerd starts on the wrong filesystem”, not data loss.
/var/lib/docker must be on /storage, and docker info will not tell you it is not
A hard requirement, not a preference. The smolvm rootfs overlay uses the initramfs (ramfs) as its lower layer, ramfs has no file-handle support, and overlayfs then rejects it as an upper dir for Docker’s nested overlay.
docker info succeeds while /var/lib/docker sits on the overlay, and the failure that
follows is confusing and much later. Assert the backing device:
smolvm machine exec --name <n> -- sh -c 'df /var/lib/docker | tail -1'
# /dev/vda 20623316 412 20606520 0% /storage Both readings print /var/lib/docker as the mount point, so the path tells you nothing. The
device is the value that matters.
Both bind mounts are needed, not just Docker’s
/var/lib/containerd holds the snapshotter’s overlay state and fails the same way if it is left
on the rootfs overlay. The upstream Smolfile mounts both and its own comments say container start
fails without the second one, with containerd’s overlay mount rejected as invalid argument.
The dockerd start command in the docs site’s guide (guides/docker-in-a-machine.md in smol-machines/docs, at v1.14.6) bind-mounts only /storage/docker; /storage/containerd is
needed as well.
A docker run that pulls interleaves two streams
The first run of an image prints the pull’s progress alongside the container’s own output, and the
two arrive in a different order on different hosts: on Linux aarch64 the marker was the last line,
on macOS arm64 it was not. A check that takes the last line passes on one host and fails on the
other. scripts/verify-docker.sh pulls first, separately, so the run’s output stands alone
without discarding the stderr that would explain a real failure.
machine delete asks for confirmation
Pass --force in a script. From v1.17.0 a delete without it on a non-terminal stdin exits 1;
before that it exited 0 and left a 20 GiB machine behind.
Where the docs site’s guide differs from this packet, at v1.14.6
The guide is guides/docker-in-a-machine.md in smol-machines/docs, published at
smolmachines.com/docs.
- Its
dockerdcommand bind-mounts one path; the upstream Smolfile mounts two. - It runs
machine create ... -s examples/docker-in-vm/docker.smolfileafter agit cloneof the smolvm repo. The release tarball does not containexamples/, so this packet ships its ownassets/docker.smolfile. - It says bind mounts “do not survive a stop and start, so reapply that mount before starting
dockerd”, which was verified; its example applies them ininit, which runs on the first start only. smolvm machine create --helpdescribes--initas “Run command on every VM start”, at v1.14.6 and at v1.22.2; from v1.23.7 it says once, on the machine’s first start. The docs site’sintroduction/concepts/smolfile.mdsays under “When init runs” that init runs once, on the first start, as doessmolfile.mdhere.- It has no platform note; on Windows the procedure cannot succeed. See
references/windows.md.