these 12 derivations will be built: /nix/store/93n9gf8jgcj9dwnv09z7j1gl50rrkqav-decrypt-age-secrets.drv /nix/store/2w3gv4sl8ic8lj83ifq0gp0ynl7il532-dry-activate.drv /nix/store/5qx2jw8w494izzzwiz7iza9cvfmzpc72-garage.toml.drv /nix/store/pridklh8appivm56qxbl4qi7x4z54zgp-X-Restart-Triggers-garage.drv /nix/store/cif2hglwrv6rfaxsv6qr7rykbjr7x2kj-unit-garage.service.drv /nix/store/5vix6jjg1ibaw8nhgrpq41s04krqjz81-system-units.drv /nix/store/i49pkz8hhm68xbb49h1csi8k7k0g67j0-etc.drv /nix/store/jzra8gry4sp9b2xmvg02inqgv5bb0w6z-activate.drv /nix/store/5lx6bjy5r3wb3ij21h92d15317rsl8nv-nixos-system-server-test.drv /nix/store/68gjqqhvwly5rp4m9l8xi71m326hz3cm-closure-info.drv /nix/store/l10vn3zyv8hd6rgzx9cqi7j6vsb766vh-nixos-test-driver-garage.drv /nix/store/6c5wyaad9zi7ykpvf3vvlsyh1f4lfqcb-container-test-run-garage.drv building '/nix/store/5qx2jw8w494izzzwiz7iza9cvfmzpc72-garage.toml.drv' building '/nix/store/93n9gf8jgcj9dwnv09z7j1gl50rrkqav-decrypt-age-secrets.drv' on 'ssh-ng://builder@build-x86-01.clan.lol' garage.toml> structuredAttrs is enabled garage.toml> building '/nix/store/pridklh8appivm56qxbl4qi7x4z54zgp-X-Restart-Triggers-garage.drv' building '/nix/store/cif2hglwrv6rfaxsv6qr7rykbjr7x2kj-unit-garage.service.drv' building '/nix/store/5vix6jjg1ibaw8nhgrpq41s04krqjz81-system-units.drv' building '/nix/store/i49pkz8hhm68xbb49h1csi8k7k0g67j0-etc.drv' building '/nix/store/93n9gf8jgcj9dwnv09z7j1gl50rrkqav-decrypt-age-secrets.drv' building '/nix/store/jzra8gry4sp9b2xmvg02inqgv5bb0w6z-activate.drv' building '/nix/store/2w3gv4sl8ic8lj83ifq0gp0ynl7il532-dry-activate.drv' building '/nix/store/5lx6bjy5r3wb3ij21h92d15317rsl8nv-nixos-system-server-test.drv' nixos-system-server-test> structuredAttrs is enabled building '/nix/store/68gjqqhvwly5rp4m9l8xi71m326hz3cm-closure-info.drv' closure-info> structuredAttrs is enabled building '/nix/store/l10vn3zyv8hd6rgzx9cqi7j6vsb766vh-nixos-test-driver-garage.drv' nixos-test-driver-garage> Running type check (enable/disable: config.skipTypeCheck) nixos-test-driver-garage> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck nixos-test-driver-garage> Success: no issues found in 1 source file building '/nix/store/6c5wyaad9zi7ykpvf3vvlsyh1f4lfqcb-container-test-run-garage.drv' on 'ssh-ng://builder@build-x86-01.clan.lol' building '/nix/store/6c5wyaad9zi7ykpvf3vvlsyh1f4lfqcb-container-test-run-garage.drv' container-test-run-garage> additionally exposed symbols: container-test-run-garage> server, container-test-run-garage> start_all, machines, driver, Machine, wait_for_signal container-test-run-garage> Starting server container-test-run-garage> container-test-run-garage> container-test-run-garage> <<< NixOS Stage 2 >>> container-test-run-garage> container-test-run-garage> booting system configuration /nix/store/9q785nlhf6xrfz2zpwiw55m4xx6f5rz1-nixos-system-server-test container-test-run-garage> running activation script... container-test-run-garage> setting up /etc... container-test-run-garage> 2: host0@if3: mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 container-test-run-garage> link/ether be:aa:dd:39:86:e7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 container-test-run-garage> setting up age secrets... container-test-run-garage> starting systemd... container-test-run-garage> systemd 260.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) container-test-run-garage> Detected virtualization systemd-nspawn. container-test-run-garage> Detected architecture x86-64. container-test-run-garage> Detected first boot. container-test-run-garage> Initializing machine ID from container UUID. container-test-run-garage> Applying preset policy. container-test-run-garage> Populated /etc with preset unit settings. container-test-run-garage> Queued start job for default target Multi-User System. container-test-run-garage> container-test-run-garage> container-test-run-garage> container-test-run-garage> <<< Welcome to NixOS test (x86_64) - console >>> container-test-run-garage> + systemd-run /bin/sh -c '/nix/store/61685rbaxmpigwi3z23i33sv3jyfl70q-coreutils-9.11/bin/sleep 999999999 && echo 5dc3c0ff-d547-4c45-93a4-570e5efeef34' container-test-run-garage> Running as unit: run-p369-i4082279.service; invocation ID: 78daaea6cc4c4795b0a975232a3a59c9 container-test-run-garage> To attach to container server run on the same machine that runs the test: container-test-run-garage> sudo nsenter --user --target $(\pgrep -f '^/bin/sh.*5dc3c0ff-d547-4c45-93a4-570e5efeef34') --mount --uts --ipc --net --pid --cgroup /bin/sh -c bash container-test-run-garage> container-test-run-garage> To inject external network and continue test, run: container-test-run-garage> sudo /nix/store/h3l4z7p6wny3phbckwwhy1i2g52pdnj4-python3-3.13.15/bin/python3.13 /nix/store/f960jm4nawb9zr6cb9d0vfpb7hifp96b-test-driver-0.0.1/lib/python3.13/site-packages/test_driver/inject_network.py 5dc3c0ff-d547-4c45-93a4-570e5efeef34 container-test-run-garage> + systemctl --no-pager show network-online.target container-test-run-garage> + systemctl --no-pager show garage container-test-run-garage> + systemctl status garage container-test-run-garage> + test -d /var/lib/garage/meta container-test-run-garage> + test -d /var/lib/garage/data container-test-run-garage> server: waiting for success: /nix/store/v894am61q5k6iybclqrpy9gc55sw5svz-libressl-4.2.1-nc/bin/nc -z -v 127.0.0.1 3901 container-test-run-garage> + /nix/store/v894am61q5k6iybclqrpy9gc55sw5svz-libressl-4.2.1-nc/bin/nc -z -v 127.0.0.1 3901 container-test-run-garage> Connection to 127.0.0.1 3901 port [tcp/nimsh] succeeded! container-test-run-garage> (finished: waiting for success: /nix/store/v894am61q5k6iybclqrpy9gc55sw5svz-libressl-4.2.1-nc/bin/nc -z -v 127.0.0.1 3901, in 0.00 seconds) container-test-run-garage> + /nix/store/v894am61q5k6iybclqrpy9gc55sw5svz-libressl-4.2.1-nc/bin/nc -z -v 127.0.0.1 3900 container-test-run-garage> Connection to 127.0.0.1 3900 port [tcp/udt-os] succeeded! container-test-run-garage> + /nix/store/v894am61q5k6iybclqrpy9gc55sw5svz-libressl-4.2.1-nc/bin/nc -z -v 127.0.0.1 3902 container-test-run-garage> Connection to 127.0.0.1 3902 port [tcp/nimaux] succeeded! container-test-run-garage> + /nix/store/v894am61q5k6iybclqrpy9gc55sw5svz-libressl-4.2.1-nc/bin/nc -z -v 127.0.0.1 3903 container-test-run-garage> Connection to 127.0.0.1 3903 port [tcp/charsetmgr] succeeded! container-test-run-garage> upload niks3: failed time=2026-09-10T00:34:58.594Z level=WARN msg="Request returned retryable status, retrying" attempt=1 max_attempts=6 backoff=100ms status=502 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T00:36:03.546Z level=WARN msg="Request returned retryable status, retrying" attempt=2 max_attempts=6 backoff=187.155328ms status=502 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T00:41:38.531Z level=WARN msg="Request returned retryable status, retrying" attempt=1 max_attempts=6 backoff=100ms status=504 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T00:46:38.632Z level=WARN msg="Request returned retryable status, retrying" attempt=2 max_attempts=6 backoff=182.285456ms status=504 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T00:52:05.427Z level=WARN msg="Request returned retryable status, retrying" attempt=1 max_attempts=6 backoff=100ms status=504 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T00:57:05.530Z level=WARN msg="Request returned retryable status, retrying" attempt=2 max_attempts=6 backoff=216.110839ms status=504 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T01:02:05.748Z level=WARN msg="Request returned retryable status, retrying" attempt=3 max_attempts=6 backoff=374.38175ms status=504 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T01:07:06.123Z level=WARN msg="Request returned retryable status, retrying" attempt=4 max_attempts=6 backoff=789.765369ms status=504 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T01:12:06.915Z level=WARN msg="Request returned retryable status, retrying" attempt=5 max_attempts=6 backoff=1.610297597s status=504 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T01:17:08.527Z level=ERROR msg="Fatal error" error="pushing paths: creating pending closures: creating pending closure: server returned 504: "