these 18 derivations will be built: /nix/store/b7rav1zyngpd23m0hq6jqabk7ka5x5pq-system-units.drv /nix/store/94vd8h6a39gqrvcf34xz94nqygab48f0-etc.drv /nix/store/rpm19j1n0s3zy6py5q9kc9zp6wayppx9-activate.drv /nix/store/73j0l1rxwhj1p9yywwqp9r1pa7d59krd-nixos-system-machine2-test.drv /nix/store/mfxi13vh37skrbwghf24hs75cpim1vl3-system-path.drv /nix/store/s2xjgcdj8y5f5lhsmk082lnw1ibnw8k1-dbus-1.drv /nix/store/hc7x9b8113za198zz9v6kmj2rjh5ci1m-X-Restart-Triggers-dbus-broker.drv /nix/store/c2qrwzgsbggj30g2qkpb50qc483z2qy3-unit-dbus-broker.service.drv /nix/store/s2clqw0aab1jzlgwrh36qzxzblf8nxb4-unit-dbus-broker.service.drv /nix/store/hazlr0xisqq1ayc194jslyy048ixlzj7-user-units.drv /nix/store/qc987sksfa02hmc89pq2fh548fj8j7da-system-units.drv /nix/store/pm4rlqdlrh9n8mnxpz0lq4pxz8vvagm3-etc.drv /nix/store/yv5wvzw0nx6f9y2m7rdx5ra3mvnysj5p-activate.drv /nix/store/gj890y72c536hfkq1s33qr6k9pwpfmld-nixos-system-machine1-test.drv /nix/store/gsk3v6j5v4m6zv28sf93f35n5q1r3l0l-closure-info.drv /nix/store/y4mxlbivhs0wvvryd8qr9s7j0h0fclq8-closure-info.drv /nix/store/zyidjcjll6drnwcxjm0a0lx30k7y3knb-nixos-test-driver-monitoring.drv /nix/store/y1yjhjz8nhkvr6r95xv8srqf197qgbi1-container-test-run-monitoring.drv building '/nix/store/mfxi13vh37skrbwghf24hs75cpim1vl3-system-path.drv' building '/nix/store/b7rav1zyngpd23m0hq6jqabk7ka5x5pq-system-units.drv' system-path> structuredAttrs is enabled system-path> created 1965 symlinks in user environment building '/nix/store/s2xjgcdj8y5f5lhsmk082lnw1ibnw8k1-dbus-1.drv' building '/nix/store/hc7x9b8113za198zz9v6kmj2rjh5ci1m-X-Restart-Triggers-dbus-broker.drv' building '/nix/store/c2qrwzgsbggj30g2qkpb50qc483z2qy3-unit-dbus-broker.service.drv' building '/nix/store/s2clqw0aab1jzlgwrh36qzxzblf8nxb4-unit-dbus-broker.service.drv' building '/nix/store/94vd8h6a39gqrvcf34xz94nqygab48f0-etc.drv' building '/nix/store/qc987sksfa02hmc89pq2fh548fj8j7da-system-units.drv' building '/nix/store/hazlr0xisqq1ayc194jslyy048ixlzj7-user-units.drv' building '/nix/store/rpm19j1n0s3zy6py5q9kc9zp6wayppx9-activate.drv' building '/nix/store/73j0l1rxwhj1p9yywwqp9r1pa7d59krd-nixos-system-machine2-test.drv' nixos-system-machine2-test> structuredAttrs is enabled building '/nix/store/y4mxlbivhs0wvvryd8qr9s7j0h0fclq8-closure-info.drv' building '/nix/store/pm4rlqdlrh9n8mnxpz0lq4pxz8vvagm3-etc.drv' closure-info> structuredAttrs is enabled building '/nix/store/yv5wvzw0nx6f9y2m7rdx5ra3mvnysj5p-activate.drv' building '/nix/store/gj890y72c536hfkq1s33qr6k9pwpfmld-nixos-system-machine1-test.drv' nixos-system-machine1-test> structuredAttrs is enabled building '/nix/store/gsk3v6j5v4m6zv28sf93f35n5q1r3l0l-closure-info.drv' closure-info> structuredAttrs is enabled building '/nix/store/zyidjcjll6drnwcxjm0a0lx30k7y3knb-nixos-test-driver-monitoring.drv' nixos-test-driver-monitoring> Running type check (enable/disable: config.skipTypeCheck) nixos-test-driver-monitoring> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck nixos-test-driver-monitoring> Success: no issues found in 1 source file building '/nix/store/y1yjhjz8nhkvr6r95xv8srqf197qgbi1-container-test-run-monitoring.drv' on 'ssh-ng://builder@build-x86-01.clan.lol' building '/nix/store/y1yjhjz8nhkvr6r95xv8srqf197qgbi1-container-test-run-monitoring.drv' container-test-run-monitoring> additionally exposed symbols: container-test-run-monitoring> machine1, machine2, container-test-run-monitoring> start_all, machines, driver, Machine, wait_for_signal container-test-run-monitoring> Starting machine1 container-test-run-monitoring> Starting machine2 container-test-run-monitoring> container-test-run-monitoring> container-test-run-monitoring> container-test-run-monitoring> <<< NixOS Stage 2 >>> container-test-run-monitoring> container-test-run-monitoring> booting system configuration /nix/store/3pmjkkm845k2i1drj6hapdyy40jam5lr-nixos-system-machine1-test container-test-run-monitoring> running activation script... container-test-run-monitoring> setting up /etc... container-test-run-monitoring> 2: host0@if3: mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 container-test-run-monitoring> link/ether 5a:94:bb:65:1f:55 brd ff:ff:ff:ff:ff:ff link-netnsid 0 container-test-run-monitoring> setting up age secrets... container-test-run-monitoring> starting systemd... container-test-run-monitoring> systemd 260.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) container-test-run-monitoring> Detected virtualization systemd-nspawn. container-test-run-monitoring> Detected architecture x86-64. container-test-run-monitoring> Detected first boot. container-test-run-monitoring> Initializing machine ID from container UUID. container-test-run-monitoring> Applying preset policy. container-test-run-monitoring> Populated /etc with preset unit settings. container-test-run-monitoring> Queued start job for default target Multi-User System. container-test-run-monitoring> container-test-run-monitoring> container-test-run-monitoring> container-test-run-monitoring> <<< Welcome to NixOS test (x86_64) - console >>> container-test-run-monitoring> + systemd-run /bin/sh -c '/nix/store/61685rbaxmpigwi3z23i33sv3jyfl70q-coreutils-9.11/bin/sleep 999999999 && echo 6ae64488-af96-43f1-8c12-d7c76374a8a4' container-test-run-monitoring> Running as unit: run-p525-i4257999.service; invocation ID: 4be9b5c1b0e84b9680eebfeaafedc227 container-test-run-monitoring> To attach to container machine1 run on the same machine that runs the test: container-test-run-monitoring> sudo nsenter --user --target $(\pgrep -f '^/bin/sh.*6ae64488-af96-43f1-8c12-d7c76374a8a4') --mount --uts --ipc --net --pid --cgroup /bin/sh -c bash container-test-run-monitoring> container-test-run-monitoring> To inject external network and continue test, run: container-test-run-monitoring> sudo /nix/store/h3l4z7p6wny3phbckwwhy1i2g52pdnj4-python3-3.13.15/bin/python3.13 /nix/store/f960jm4nawb9zr6cb9d0vfpb7hifp96b-test-driver-0.0.1/lib/python3.13/site-packages/test_driver/inject_network.py 6ae64488-af96-43f1-8c12-d7c76374a8a4 container-test-run-monitoring> container-test-run-monitoring> <<< NixOS Stage 2 >>> container-test-run-monitoring> container-test-run-monitoring> booting system configuration /nix/store/5zlsjkyklxnkyqjj5h3j5kjwcg0fii4s-nixos-system-machine2-test container-test-run-monitoring> running activation script... container-test-run-monitoring> setting up /etc... container-test-run-monitoring> 2: host0@if4: mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 container-test-run-monitoring> link/ether 46:8d:8e:53:39:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 container-test-run-monitoring> setting up age secrets... container-test-run-monitoring> starting systemd... container-test-run-monitoring> systemd 260.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) container-test-run-monitoring> Detected virtualization systemd-nspawn. container-test-run-monitoring> Detected architecture x86-64. container-test-run-monitoring> Detected first boot. container-test-run-monitoring> Initializing machine ID from container UUID. container-test-run-monitoring> Applying preset policy. container-test-run-monitoring> Populated /etc with preset unit settings. container-test-run-monitoring> Queued start job for default target Multi-User System. container-test-run-monitoring> container-test-run-monitoring> container-test-run-monitoring> container-test-run-monitoring> <<< Welcome to NixOS test (x86_64) - console >>> container-test-run-monitoring> + systemd-run /bin/sh -c '/nix/store/61685rbaxmpigwi3z23i33sv3jyfl70q-coreutils-9.11/bin/sleep 999999999 && echo 62919cf1-07a4-4ef5-b3ad-7a613d665a56' container-test-run-monitoring> Running as unit: run-p365-i4258002.service; invocation ID: 08fc37661a7a416284942f6197e0768f container-test-run-monitoring> To attach to container machine2 run on the same machine that runs the test: container-test-run-monitoring> sudo nsenter --user --target $(\pgrep -f '^/bin/sh.*62919cf1-07a4-4ef5-b3ad-7a613d665a56') --mount --uts --ipc --net --pid --cgroup /bin/sh -c bash container-test-run-monitoring> container-test-run-monitoring> To inject external network and continue test, run: container-test-run-monitoring> sudo /nix/store/h3l4z7p6wny3phbckwwhy1i2g52pdnj4-python3-3.13.15/bin/python3.13 /nix/store/f960jm4nawb9zr6cb9d0vfpb7hifp96b-test-driver-0.0.1/lib/python3.13/site-packages/test_driver/inject_network.py 62919cf1-07a4-4ef5-b3ad-7a613d665a56 container-test-run-monitoring> + systemctl --no-pager show multi-user.target container-test-run-monitoring> + systemctl list-jobs --full container-test-run-monitoring> + systemctl --no-pager show multi-user.target container-test-run-monitoring> + systemctl list-jobs --full container-test-run-monitoring> + systemctl --no-pager show multi-user.target container-test-run-monitoring> + systemctl --no-pager show multi-user.target container-test-run-monitoring> + systemctl --no-pager show alloy container-test-run-monitoring> + systemctl --no-pager show alloy container-test-run-monitoring> + test -f /etc/alloy/config.alloy container-test-run-monitoring> + test -f /etc/alloy/config.alloy container-test-run-monitoring> + test -f /etc/alloy/local-extension.alloy container-test-run-monitoring> + systemctl show alloy --property=ExecStart container-test-run-monitoring> + grep -F /etc/alloy container-test-run-monitoring> + systemctl show alloy --property=ExecStart container-test-run-monitoring> + grep -F /etc/alloy container-test-run-monitoring> + systemctl --no-pager show loki container-test-run-monitoring> + systemctl --no-pager show mimir container-test-run-monitoring> + systemctl --no-pager show grafana container-test-run-monitoring> ++ grep -c '^loki.source.journal ' /etc/alloy/config.alloy container-test-run-monitoring> + test 1 = 1 container-test-run-monitoring> + cat /etc/alloy/config.alloy container-test-run-monitoring> container-test-run-monitoring> upload niks3: failed time=2026-09-10T00:34:58.594Z level=WARN msg="Request returned retryable status, retrying" attempt=1 max_attempts=6 backoff=100ms status=502 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T00:36:03.546Z level=WARN msg="Request returned retryable status, retrying" attempt=2 max_attempts=6 backoff=187.155328ms status=502 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T00:41:38.531Z level=WARN msg="Request returned retryable status, retrying" attempt=1 max_attempts=6 backoff=100ms status=504 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T00:46:38.632Z level=WARN msg="Request returned retryable status, retrying" attempt=2 max_attempts=6 backoff=182.285456ms status=504 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T00:52:05.427Z level=WARN msg="Request returned retryable status, retrying" attempt=1 max_attempts=6 backoff=100ms status=504 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T00:57:05.530Z level=WARN msg="Request returned retryable status, retrying" attempt=2 max_attempts=6 backoff=216.110839ms status=504 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T01:02:05.748Z level=WARN msg="Request returned retryable status, retrying" attempt=3 max_attempts=6 backoff=374.38175ms status=504 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T01:07:06.123Z level=WARN msg="Request returned retryable status, retrying" attempt=4 max_attempts=6 backoff=789.765369ms status=504 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T01:12:06.915Z level=WARN msg="Request returned retryable status, retrying" attempt=5 max_attempts=6 backoff=1.610297597s status=504 url=https://niks3.clan.lol/api/pending_closures time=2026-09-10T01:17:08.527Z level=ERROR msg="Fatal error" error="pushing paths: creating pending closures: creating pending closure: server returned 504: "