these 51 derivations will be built: /nix/store/0jafmcvdjhmdspjv03s47nw7rg9bb4n8-vars-check-monitoring.drv /nix/store/lmcvq1ng5qlgmii82c9dw9ymmxw93j6a-dashboard.yaml.drv /nix/store/invz7086xlsvm0sf35x20h9q18j4hc4y-grafana-provisioning.drv /nix/store/2gpgg7rcd9b75g9v8qn3mzmgp28na2x5-config.ini.drv /nix/store/565j0l3d6ld5arm9d1xawcmckx9yrida-unit-grafana.service.drv /nix/store/gpihxkhycha1iybsaldb25sxlhwpqzsd-loki-config.json.drv /nix/store/4j2rmp6l85yxrdll17x5r0ry02yx3pp9-validate-loki-conf.drv /nix/store/80raxhif8334ji2v4zjw6pm1cfbqh8v8-unit-loki.service.drv /nix/store/7n2vl81wqhvhw0vyv5qzkzywfb2lzqkf-nginx.conf.drv /nix/store/40g10gpr175dv0574c449w8xlfpgpajv-unit-script-nginx-pre-start.drv /nix/store/8c60d8qklvwr3jvh8rglfw837084lkkk-unit-nginx.service.drv /nix/store/9cw9km5qsfskdkxc3rc9kz6gfxakaf9a-config.yaml.drv /nix/store/bhnq91370bayzkc91d47628ixpy84p53-unit-mimir.service.drv /nix/store/6p4dwlybf23all5f8xhd92gc0zgi5s2f-X-Reload-Triggers-alloy.drv /nix/store/ffm8wxlvn5f8mhcxj2hrd6631w09yz84-unit-alloy.service.drv /nix/store/hi8cpa4yy19qr9vvyn0p9gr1zs86mh6y-firewall-start.drv /nix/store/iy5r8sagnvvxndvcx5wf93r4plh1gw7q-firewall-reload.drv /nix/store/hpg303h299194nb31hyrmz5ijllvfc0r-unit-firewall.service.drv /nix/store/gh90p6znai49hv0x2rz8m08jh5gfr4sg-system-path.drv /nix/store/927s52zxwx74myjspbwkz3crjb9618m2-dbus-1.drv /nix/store/143s3yrc39mycl5zm051nw2anw4lxgaf-X-Restart-Triggers-dbus-broker.drv /nix/store/p8iq9h4xc23zp39bpwar30bz4lra1niy-unit-dbus-broker.service.drv /nix/store/0pgnqwisrvcfmnpfqgw4x2k65a6kc6k1-system-units.drv /nix/store/6bphpms7a4695inazklvkc53y7n1fqvr-string-hosts.drv /nix/store/g1c0ifkz66r45s8rygksfm6a9lnmhahi-extra-hosts.drv /nix/store/0s7n87b93c8ccx2fdpsspc1wvqhs8j0i-hosts.drv /nix/store/0x3cav0sb1jqjpywgk9148176zn53xj1-decrypt-age-secrets.drv /nix/store/ql0q0xi80ynpi6vpv43jlxhr0s26c7bh-string-hosts.drv /nix/store/1d23wyklwa0a17x3iaq1s6757diqd8d4-hosts.drv /nix/store/mcqr1b7wixr4dlhnimz7d6yvl28jb4p0-etc-alloy-local-extension.alloy.drv /nix/store/mvwm7ckf0hpms0l3lvbpaspcxg7z8f5w-X-Reload-Triggers-alloy.drv /nix/store/2rjg3q1m0zzifkm7ri816h37q5mddf5v-unit-alloy.service.drv /nix/store/5y0j2hqjl91hzgrqwnzp6qg7dvn2wp2d-decrypt-age-secrets.drv /nix/store/2ybd7rc0g1gspc8dcra5zafiimfwjnqw-dry-activate.drv /nix/store/b8xv1f0i55l16faprkkas0h0b2gqwkjs-system-units.drv /nix/store/bb7ync4j87h7p8aar5mszd4rkl1glw53-etc.drv /nix/store/z13qypbjddvq590xb8lhyp3cz10rqb8k-activate.drv /nix/store/rbydp6lb42wwd6m17caq7pwqg1qzghi9-nixos-system-machine2-test.drv /nix/store/57gcgxyqns8d9lm8ra2p45p71qszw76z-run-machine2-nspawn.drv /nix/store/qagrgzbb651mbdzb1l37fgdj3zrm3b90-users-groups.json.drv /nix/store/sqlwbkd7mp4bb6dknf90rhvpgdksy5sy-unit-dbus-broker.service.drv /nix/store/nq1dvilsvnjmzi3snab38is5wygi65qd-user-units.drv /nix/store/x315pw8i8m6naz67qrcyghsi8rsbkxj4-etc.drv /nix/store/hdkcxgchmbakr5f3imwj9d829jmm96y1-activate.drv /nix/store/zkxxwxdjwks8ghzgc4665fffl39bav2m-dry-activate.drv /nix/store/ic8ljf0f3f10g9hq1jvvn8g6gnwd6sh9-nixos-system-machine1-test.drv /nix/store/c6bpy2h26ly16b5qzzwr3nxnpm1gvz2r-run-machine1-nspawn.drv /nix/store/lvji31yh2c417ypazf9qh3qvbyri81cd-test-script.drv /nix/store/anm3ip7qp4qnalghkbpj1x70xj546bg3-driverConfiguration.json.drv /nix/store/ihxa014vlbjacl2lblx8xihzfbwk4a9m-nixos-test-driver-monitoring.drv /nix/store/3hdiiv6cfh7l5mqxp3rcrxiih1lk6rv3-container-test-run-monitoring.drv building '/nix/store/lvji31yh2c417ypazf9qh3qvbyri81cd-test-script.drv' building '/nix/store/gh90p6znai49hv0x2rz8m08jh5gfr4sg-system-path.drv' building '/nix/store/mcqr1b7wixr4dlhnimz7d6yvl28jb4p0-etc-alloy-local-extension.alloy.drv' building '/nix/store/7n2vl81wqhvhw0vyv5qzkzywfb2lzqkf-nginx.conf.drv' building '/nix/store/g1c0ifkz66r45s8rygksfm6a9lnmhahi-extra-hosts.drv' building '/nix/store/6bphpms7a4695inazklvkc53y7n1fqvr-string-hosts.drv' building '/nix/store/ql0q0xi80ynpi6vpv43jlxhr0s26c7bh-string-hosts.drv' building '/nix/store/6p4dwlybf23all5f8xhd92gc0zgi5s2f-X-Reload-Triggers-alloy.drv' building '/nix/store/hi8cpa4yy19qr9vvyn0p9gr1zs86mh6y-firewall-start.drv' X-Reload-Triggers-alloy> structuredAttrs is enabled etc-alloy-local-extension.alloy> structuredAttrs is enabled extra-hosts> structuredAttrs is enabled nginx.conf> structuredAttrs is enabled string-hosts> structuredAttrs is enabled string-hosts> structuredAttrs is enabled system-path> structuredAttrs is enabled system-path> created 1873 symlinks in user environment test-script> structuredAttrs is enabled building '/nix/store/mvwm7ckf0hpms0l3lvbpaspcxg7z8f5w-X-Reload-Triggers-alloy.drv' building '/nix/store/0s7n87b93c8ccx2fdpsspc1wvqhs8j0i-hosts.drv' building '/nix/store/1d23wyklwa0a17x3iaq1s6757diqd8d4-hosts.drv' building '/nix/store/ffm8wxlvn5f8mhcxj2hrd6631w09yz84-unit-alloy.service.drv' X-Reload-Triggers-alloy> structuredAttrs is enabled firewall-start> structuredAttrs is enabled nginx.conf> nginx.conf> ==================== Results =================== nginx.conf> No issues found. nginx.conf> nginx.conf> ==================== Summary =================== nginx.conf> Total issues: nginx.conf> Unspecified: 0 nginx.conf> Low: 0 nginx.conf> Medium: 0 nginx.conf> High: 0 nginx.conf> building '/nix/store/927s52zxwx74myjspbwkz3crjb9618m2-dbus-1.drv' building '/nix/store/9cw9km5qsfskdkxc3rc9kz6gfxakaf9a-config.yaml.drv' building '/nix/store/0jafmcvdjhmdspjv03s47nw7rg9bb4n8-vars-check-monitoring.drv' building '/nix/store/qagrgzbb651mbdzb1l37fgdj3zrm3b90-users-groups.json.drv' config.yaml> structuredAttrs is enabled unit-alloy.service> structuredAttrs is enabled vars-check-monitoring> Running vars check using Nix-based executor... vars-check-monitoring> ✓ All vars checks completed successfully building '/nix/store/143s3yrc39mycl5zm051nw2anw4lxgaf-X-Restart-Triggers-dbus-broker.drv' building '/nix/store/iy5r8sagnvvxndvcx5wf93r4plh1gw7q-firewall-reload.drv' building '/nix/store/2rjg3q1m0zzifkm7ri816h37q5mddf5v-unit-alloy.service.drv' building '/nix/store/40g10gpr175dv0574c449w8xlfpgpajv-unit-script-nginx-pre-start.drv' X-Restart-Triggers-dbus-broker> structuredAttrs is enabled firewall-reload> structuredAttrs is enabled unit-alloy.service> structuredAttrs is enabled users-groups.json> structuredAttrs is enabled config.yaml> unit-script-nginx-pre-start> structuredAttrs is enabled building '/nix/store/b8xv1f0i55l16faprkkas0h0b2gqwkjs-system-units.drv' building '/nix/store/p8iq9h4xc23zp39bpwar30bz4lra1niy-unit-dbus-broker.service.drv' building '/nix/store/sqlwbkd7mp4bb6dknf90rhvpgdksy5sy-unit-dbus-broker.service.drv' building '/nix/store/hpg303h299194nb31hyrmz5ijllvfc0r-unit-firewall.service.drv' building '/nix/store/bhnq91370bayzkc91d47628ixpy84p53-unit-mimir.service.drv' building '/nix/store/lmcvq1ng5qlgmii82c9dw9ymmxw93j6a-dashboard.yaml.drv' building '/nix/store/0x3cav0sb1jqjpywgk9148176zn53xj1-decrypt-age-secrets.drv' building '/nix/store/5y0j2hqjl91hzgrqwnzp6qg7dvn2wp2d-decrypt-age-secrets.drv' building '/nix/store/gpihxkhycha1iybsaldb25sxlhwpqzsd-loki-config.json.drv' dashboard.yaml> structuredAttrs is enabled decrypt-age-secrets> structuredAttrs is enabled decrypt-age-secrets> structuredAttrs is enabled unit-dbus-broker.service> structuredAttrs is enabled unit-dbus-broker.service> structuredAttrs is enabled unit-firewall.service> structuredAttrs is enabled unit-mimir.service> structuredAttrs is enabled building '/nix/store/8c60d8qklvwr3jvh8rglfw837084lkkk-unit-nginx.service.drv' building '/nix/store/2ybd7rc0g1gspc8dcra5zafiimfwjnqw-dry-activate.drv' building '/nix/store/zkxxwxdjwks8ghzgc4665fffl39bav2m-dry-activate.drv' building '/nix/store/nq1dvilsvnjmzi3snab38is5wygi65qd-user-units.drv' building '/nix/store/4j2rmp6l85yxrdll17x5r0ry02yx3pp9-validate-loki-conf.drv' dashboard.yaml> dry-activate> structuredAttrs is enabled dry-activate> structuredAttrs is enabled unit-nginx.service> structuredAttrs is enabled building '/nix/store/invz7086xlsvm0sf35x20h9q18j4hc4y-grafana-provisioning.drv' validate-loki-conf> level=info ts=2026-09-13T15:09:57.903476389Z caller=main.go:109 msg="config is valid" building '/nix/store/2gpgg7rcd9b75g9v8qn3mzmgp28na2x5-config.ini.drv' building '/nix/store/bb7ync4j87h7p8aar5mszd4rkl1glw53-etc.drv' config.ini> structuredAttrs is enabled building '/nix/store/80raxhif8334ji2v4zjw6pm1cfbqh8v8-unit-loki.service.drv' building '/nix/store/565j0l3d6ld5arm9d1xawcmckx9yrida-unit-grafana.service.drv' unit-loki.service> structuredAttrs is enabled unit-grafana.service> structuredAttrs is enabled building '/nix/store/0pgnqwisrvcfmnpfqgw4x2k65a6kc6k1-system-units.drv' building '/nix/store/z13qypbjddvq590xb8lhyp3cz10rqb8k-activate.drv' activate> structuredAttrs is enabled building '/nix/store/rbydp6lb42wwd6m17caq7pwqg1qzghi9-nixos-system-machine2-test.drv' nixos-system-machine2-test> structuredAttrs is enabled building '/nix/store/57gcgxyqns8d9lm8ra2p45p71qszw76z-run-machine2-nspawn.drv' building '/nix/store/x315pw8i8m6naz67qrcyghsi8rsbkxj4-etc.drv' building '/nix/store/hdkcxgchmbakr5f3imwj9d829jmm96y1-activate.drv' activate> structuredAttrs is enabled building '/nix/store/ic8ljf0f3f10g9hq1jvvn8g6gnwd6sh9-nixos-system-machine1-test.drv' nixos-system-machine1-test> structuredAttrs is enabled building '/nix/store/c6bpy2h26ly16b5qzzwr3nxnpm1gvz2r-run-machine1-nspawn.drv' building '/nix/store/anm3ip7qp4qnalghkbpj1x70xj546bg3-driverConfiguration.json.drv' driverConfiguration.json> structuredAttrs is enabled building '/nix/store/ihxa014vlbjacl2lblx8xihzfbwk4a9m-nixos-test-driver-monitoring.drv' nixos-test-driver-monitoring> Running type check (enable/disable: config.skipTypeCheck) nixos-test-driver-monitoring> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck nixos-test-driver-monitoring> All checks passed! nixos-test-driver-monitoring> Linting test script (enable/disable: config.skipLint) nixos-test-driver-monitoring> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint nixos-test-driver-monitoring> All checks passed! building '/nix/store/3hdiiv6cfh7l5mqxp3rcrxiih1lk6rv3-container-test-run-monitoring.drv' container-test-run-monitoring> Machine state will be reset. To keep it, pass --keep-machine-state container-test-run-monitoring> start all VLans container-test-run-monitoring> (finished: start all VLans, in 0.00 seconds) container-test-run-monitoring> container-test-run-monitoring> Test will time out and terminate in 3600.0 seconds container-test-run-monitoring> run the VM test script container-test-run-monitoring> additionally exposed symbols: container-test-run-monitoring> machine1, machine2, container-test-run-monitoring> vlan1, container-test-run-monitoring> start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh container-test-run-monitoring> start all VMs container-test-run-monitoring> machine1: systemd-nspawn running (pid 52) container-test-run-monitoring> machine2: systemd-nspawn running (pid 53) container-test-run-monitoring> machine1: Waiting for journal at /build/vm-state-machine1/var/log/journal... container-test-run-monitoring> machine2: Waiting for journal at /build/vm-state-machine2/var/log/journal... container-test-run-monitoring> (finished: start all VMs, in 0.00 seconds) container-test-run-monitoring> machine1: waiting for unit multi-user.target container-test-run-monitoring> nixos-nspawn(machine2): TAP vde-tap1 not found; container will be isolated from VDE container-test-run-monitoring> nixos-nspawn(machine2): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths. container-test-run-monitoring> nixos-nspawn(machine1): TAP vde-tap1 not found; container will be isolated from VDE container-test-run-monitoring> nixos-nspawn(machine1): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths. container-test-run-monitoring> Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file. container-test-run-monitoring> Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file. container-test-run-monitoring> ░ Spawning container machine2 on /build/vm-state-machine2. container-test-run-monitoring> ░ Spawning container machine1 on /build/vm-state-machine1. container-test-run-monitoring> machine1 # [805053.664031] machine1 systemd-journald[126]: Journal started container-test-run-monitoring> machine1 # [805053.664086] machine1 systemd-journald[126]: Runtime Journal (/run/log/journal/c84081f8588a494eac096db5098ae543) is 8M, max 4G, 3.9G free. container-test-run-monitoring> machine1 # [805053.695625] machine1 systemd[1]: Finished Apply Kernel Variables. container-test-run-monitoring> machine1 # [805053.713969] machine1 systemd[1]: Finished Create Static Device Nodes in /dev gracefully. container-test-run-monitoring> machine1 # [805053.729293] machine1 systemd[1]: Starting Flush Journal to Persistent Storage... container-test-run-monitoring> machine1 # [805053.730289] machine1 systemd[1]: Starting Network Name Resolution... container-test-run-monitoring> machine1 # [805053.730724] machine1 systemd[1]: Starting Create Static Device Nodes in /dev... container-test-run-monitoring> machine1 # [805053.767227] machine1 systemd-journald[126]: Time spent on flushing to /var/log/journal/c84081f8588a494eac096db5098ae543 is 1.479ms for 7 entries. container-test-run-monitoring> machine2 # [805053.663568] machine2 systemd-journald[108]: Journal started container-test-run-monitoring> machine2 # [805053.663634] machine2 systemd-journald[108]: Runtime Journal (/run/log/journal/a7131fb525854959acfad912e5f02d5c) is 8M, max 4G, 3.9G free. container-test-run-monitoring> machine2 # [805053.693117] machine2 systemd[1]: Finished Apply Kernel Variables. container-test-run-monitoring> machine2 # [805053.711807] machine2 systemd[1]: Finished Create Static Device Nodes in /dev gracefully. container-test-run-monitoring> machine2 # [805053.730950] machine2 systemd[1]: Starting Flush Journal to Persistent Storage... container-test-run-monitoring> machine2 # [805053.731556] machine2 systemd[1]: Starting Network Name Resolution... container-test-run-monitoring> machine2 # [805053.732771] machine2 systemd[1]: Starting Create Static Device Nodes in /dev... container-test-run-monitoring> machine2 # [805053.767243] machine2 systemd-journald[108]: Time spent on flushing to /var/log/journal/a7131fb525854959acfad912e5f02d5c is 1.717ms for 7 entries. container-test-run-monitoring> machine2 # [805053.767243] machine2 systemd-journald[108]: System Journal (/var/log/journal/a7131fb525854959acfad912e5f02d5c) is 512B, max 4G, 3.9G free. container-test-run-monitoring> machine2 # [805053.799857] machine2 systemd[1]: Finished Firewall. container-test-run-monitoring> machine2 # [805053.799990] machine2 systemd[1]: Finished Flush Journal to Persistent Storage. container-test-run-monitoring> machine2 # [805053.800420] machine2 systemd[1]: Finished Create Static Device Nodes in /dev. container-test-run-monitoring> machine2 # [805053.801890] machine2 systemd[1]: Reached target Preparation for Local File Systems. container-test-run-monitoring> machine2 # [805053.801976] machine2 systemd[1]: Reached target Local File Systems. container-test-run-monitoring> machine2 # [805053.802033] machine2 systemd[1]: Reached target Preparation for Network. container-test-run-monitoring> machine2 # [805053.802658] machine2 systemd[1]: Listening on Boot Loader Control Service Socket. container-test-run-monitoring> machine2 # [805053.802802] machine2 systemd[1]: Listening on Network Management Resolve Hook Socket. container-test-run-monitoring> machine2 # [805053.802831] machine2 systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container container-test-run-monitoring> machine2 # [805053.803381] machine2 systemd[1]: Starting Save Transient machine-id to Disk... container-test-run-monitoring> machine2 # [805053.803803] machine2 systemd[1]: Starting Create System Files and Directories... container-test-run-monitoring> machine2 # [805053.803821] machine2 systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys container-test-run-monitoring> machine2 # [805053.804455] machine2 systemd[1]: Starting Network Management... container-test-run-monitoring> machine2 # [805053.833751] machine2 systemd-tmpfiles[213]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted container-test-run-monitoring> machine2 # [805053.833964] machine2 systemd-tmpfiles[213]: fchmod() of /var/log/journal failed: Operation not permitted container-test-run-monitoring> machine2 # [805053.834127] machine2 systemd-tmpfiles[213]: fchmod() of /var/log/journal/a7131fb525854959acfad912e5f02d5c failed: Operation not permitted container-test-run-monitoring> machine1 # [805053.767227] machine1 systemd-journald[126]: System Journal (/var/log/journal/c84081f8588a494eac096db5098ae543) is 512B, max 4G, 3.9G free. container-test-run-monitoring> machine1 # [805053.799559] machine1 systemd[1]: Finished Firewall. container-test-run-monitoring> machine1 # [805053.799748] machine1 systemd[1]: Finished Flush Journal to Persistent Storage. container-test-run-monitoring> machine1 # [805053.800034] machine1 systemd[1]: Finished Create Static Device Nodes in /dev. container-test-run-monitoring> machine1 # [805053.802297] machine1 systemd[1]: Reached target Preparation for Local File Systems. container-test-run-monitoring> machine1 # [805053.802370] machine1 systemd[1]: Reached target Local File Systems. container-test-run-monitoring> machine1 # [805053.802416] machine1 systemd[1]: Reached target Preparation for Network. container-test-run-monitoring> machine1 # [805053.803123] machine1 systemd[1]: Listening on Boot Loader Control Service Socket. container-test-run-monitoring> machine1 # [805053.803300] machine1 systemd[1]: Listening on Network Management Resolve Hook Socket. container-test-run-monitoring> machine1 # [805053.803336] machine1 systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container container-test-run-monitoring> machine1 # [805053.803915] machine1 systemd[1]: Starting Save Transient machine-id to Disk... container-test-run-monitoring> machine1 # [805053.804681] machine1 systemd[1]: Starting Create System Files and Directories... container-test-run-monitoring> machine1 # [805053.804701] machine1 systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys container-test-run-monitoring> machine1 # [805053.805248] machine1 systemd[1]: Starting Network Management... container-test-run-monitoring> machine1 # [805053.834068] machine1 systemd-tmpfiles[233]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted container-test-run-monitoring> machine1 # [805053.834294] machine1 systemd-tmpfiles[233]: fchmod() of /var/log/journal failed: Operation not permitted container-test-run-monitoring> machine1 # [805053.834448] machine1 systemd-tmpfiles[233]: fchmod() of /var/log/journal/c84081f8588a494eac096db5098ae543 failed: Operation not permitted container-test-run-monitoring> machine2 # [805053.834339] machine2 systemd-tmpfiles[213]: fchmod() of /run/log/journal failed: Operation not permitted container-test-run-monitoring> machine1 # [805053.834660] machine1 systemd-tmpfiles[233]: fchmod() of /run/log/journal failed: Operation not permitted container-test-run-monitoring> machine2 # [805053.839906] machine2 systemd[1]: Finished Create System Files and Directories. container-test-run-monitoring> machine1 # [805053.843928] machine1 systemd[1]: Finished Create System Files and Directories. container-test-run-monitoring> machine2 # [805053.840695] machine2 systemd[1]: Starting Rebuild Journal Catalog... container-test-run-monitoring> machine1 # [805053.844745] machine1 systemd[1]: Starting Rebuild Journal Catalog... container-test-run-monitoring> machine2 # [805053.841123] machine2 systemd[1]: Starting Record System Boot/Shutdown in UTMP... container-test-run-monitoring> machine1 # [805053.845210] machine1 systemd[1]: Starting Record System Boot/Shutdown in UTMP... container-test-run-monitoring> machine2 # [805053.850944] machine2 systemd[1]: Finished Record System Boot/Shutdown in UTMP. container-test-run-monitoring> machine1 # [805053.854539] machine1 systemd[1]: Finished Record System Boot/Shutdown in UTMP. container-test-run-monitoring> machine2 # [805053.858812] machine2 systemd[1]: Finished Rebuild Journal Catalog. container-test-run-monitoring> machine1 # [805053.863657] machine1 systemd[1]: Finished Rebuild Journal Catalog. container-test-run-monitoring> machine2 # [805053.859445] machine2 systemd[1]: Starting Update is Completed... container-test-run-monitoring> machine1 # [805053.864405] machine1 systemd[1]: Starting Update is Completed... container-test-run-monitoring> machine2 # [805053.866397] machine2 systemd[1]: Finished Update is Completed. container-test-run-monitoring> machine1 # [805053.872807] machine1 systemd[1]: Finished Update is Completed. container-test-run-monitoring> machine1 # [805054.028982] machine1 systemd[1]: Finished Save Transient machine-id to Disk. container-test-run-monitoring> machine2 # [805054.028993] machine2 systemd[1]: Finished Save Transient machine-id to Disk. container-test-run-monitoring> machine2 # [805054.663787] machine2 systemd[1]: etc-machine\x2did.mount: Deactivated successfully. container-test-run-monitoring> machine1 # [805054.663925] machine1 systemd[1]: etc-machine\x2did.mount: Deactivated successfully. container-test-run-monitoring> machine2 # [805054.924848] machine2 systemd-networkd[214]: Failed to increase receive buffer size for general netlink socket, ignoring: Operation not permitted container-test-run-monitoring> machine2 # [805054.924940] machine2 systemd-networkd[214]: Failed to increase receive buffer size for nftables netlink socket, ignoring: Operation not permitted container-test-run-monitoring> machine2 # [805054.937412] machine2 systemd-networkd[214]: /etc/systemd/network/99-ethernet-default-dhcp.network: No valid settings found in the [Match] section, ignoring file. To match all interfaces, add Name=* in the [Match] section. container-test-run-monitoring> machine2 # [805054.937614] machine2 systemd-networkd[214]: /etc/systemd/network/99-wireless-client-dhcp.network: No valid settings found in the [Match] section, ignoring file. To match all interfaces, add Name=* in the [Match] section. container-test-run-monitoring> machine2 # [805054.937737] machine2 systemd-networkd[214]: lo: Link UP container-test-run-monitoring> machine2 # [805054.937742] machine2 systemd-networkd[214]: lo: Gained carrier container-test-run-monitoring> machine2 # [805054.937916] machine2 systemd-networkd[214]: eth1: Configuring with /etc/systemd/network/40-eth1.network. container-test-run-monitoring> machine2 # [805054.938226] machine2 systemd[1]: Started Network Management. container-test-run-monitoring> machine2 # [805054.938317] machine2 systemd-networkd[214]: eth1: Link UP container-test-run-monitoring> machine2 # [805054.938545] machine2 systemd-networkd[214]: eth1: Gained carrier container-test-run-monitoring> machine2 # [805054.939544] machine2 systemd[1]: Starting Enable Persistent Storage in systemd-networkd... container-test-run-monitoring> machine2 # [805054.957614] machine2 systemd[1]: Finished Enable Persistent Storage in systemd-networkd. container-test-run-monitoring> machine1 # [805054.961791] machine1 systemd-networkd[234]: Failed to increase receive buffer size for general netlink socket, ignoring: Operation not permitted container-test-run-monitoring> machine1 # [805054.961884] machine1 systemd-networkd[234]: Failed to increase receive buffer size for nftables netlink socket, ignoring: Operation not permitted container-test-run-monitoring> machine1 # [805054.973800] machine1 systemd-networkd[234]: /etc/systemd/network/99-ethernet-default-dhcp.network: No valid settings found in the [Match] section, ignoring file. To match all interfaces, add Name=* in the [Match] section. container-test-run-monitoring> machine1 # [805054.974009] machine1 systemd-networkd[234]: /etc/systemd/network/99-wireless-client-dhcp.network: No valid settings found in the [Match] section, ignoring file. To match all interfaces, add Name=* in the [Match] section. container-test-run-monitoring> machine1 # [805054.974171] machine1 systemd-networkd[234]: lo: Link UP container-test-run-monitoring> machine1 # [805054.974176] machine1 systemd-networkd[234]: lo: Gained carrier container-test-run-monitoring> machine1 # [805054.974373] machine1 systemd-networkd[234]: eth1: Configuring with /etc/systemd/network/40-eth1.network. container-test-run-monitoring> machine1 # [805054.974709] machine1 systemd[1]: Started Network Management. container-test-run-monitoring> machine1 # [805054.974774] machine1 systemd-networkd[234]: eth1: Link UP container-test-run-monitoring> machine1 # [805054.975016] machine1 systemd-networkd[234]: eth1: Gained carrier container-test-run-monitoring> machine1 # [805054.975905] machine1 systemd[1]: Starting Enable Persistent Storage in systemd-networkd... container-test-run-monitoring> machine1 # [805055.007154] machine1 systemd[1]: Finished Enable Persistent Storage in systemd-networkd. container-test-run-monitoring> machine2 # [805055.654786] machine2 systemd-resolved[167]: Positive Trust Anchors: container-test-run-monitoring> machine2 # [805055.654796] machine2 systemd-resolved[167]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d container-test-run-monitoring> machine2 # [805055.654801] machine2 systemd-resolved[167]: . IN DS 38696 8 2 683d2d0acb8c9b712a1948b27f741219298d0a450d612c483af444a4c0fb2b16 container-test-run-monitoring> machine2 # [805055.654823] machine2 systemd-resolved[167]: Negative trust anchors: home.arpa 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 170.0.0.192.in-addr.arpa 171.0.0.192.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa ipv4only.arpa resolver.arpa corp home internal intranet lan local private test container-test-run-monitoring> machine2 # [805055.670029] machine2 systemd-resolved[167]: Using system hostname 'machine2'. container-test-run-monitoring> machine2 # [805055.671290] machine2 systemd[1]: Started Network Name Resolution. container-test-run-monitoring> machine2 # [805055.671371] machine2 systemd[1]: Reached target Network. container-test-run-monitoring> machine2 # [805055.671424] machine2 systemd[1]: Reached target System Initialization. container-test-run-monitoring> machine2 # [805055.671472] machine2 systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container container-test-run-monitoring> machine2 # [805055.671496] machine2 systemd[1]: Started Daily Cleanup of Temporary Directories. container-test-run-monitoring> machine2 # [805055.671511] machine2 systemd[1]: Reached target Timer Units. container-test-run-monitoring> machine2 # [805055.671641] machine2 systemd[1]: Listening on D-Bus System Message Bus Socket. container-test-run-monitoring> machine2 # [805055.671816] machine2 systemd[1]: Listening on Nix Daemon Socket. container-test-run-monitoring> machine2 # [805055.671941] machine2 systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. container-test-run-monitoring> machine2 # [805055.671959] machine2 systemd[1]: Reached target Socket Units. container-test-run-monitoring> machine2 # [805055.671988] machine2 systemd[1]: Reached target Basic System. container-test-run-monitoring> machine2 # [805055.673051] machine2 systemd[1]: Started alloy.service. container-test-run-monitoring> machine2 # [805055.673475] machine2 systemd[1]: Starting Import lastlog data into lastlog2 database... container-test-run-monitoring> machine2 # [805055.674042] machine2 systemd[1]: Starting Name Service Cache Daemon (nsncd)... container-test-run-monitoring> machine2 # [805055.674941] machine2 systemd[1]: Starting D-Bus System Message Bus... container-test-run-monitoring> machine2 # [805055.686985] machine2 systemd[1]: Finished Import lastlog data into lastlog2 database. container-test-run-monitoring> machine1 # [805055.624673] machine1 systemd-resolved[184]: Positive Trust Anchors: container-test-run-monitoring> machine1 # [805055.624684] machine1 systemd-resolved[184]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d container-test-run-monitoring> machine1 # [805055.624688] machine1 systemd-resolved[184]: . IN DS 38696 8 2 683d2d0acb8c9b712a1948b27f741219298d0a450d612c483af444a4c0fb2b16 container-test-run-monitoring> machine1 # [805055.624709] machine1 systemd-resolved[184]: Negative trust anchors: home.arpa 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 170.0.0.192.in-addr.arpa 171.0.0.192.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa ipv4only.arpa resolver.arpa corp home internal intranet lan local private test container-test-run-monitoring> machine1 # [805055.640009] machine1 systemd-resolved[184]: Using system hostname 'machine1'. container-test-run-monitoring> machine1 # [805055.641291] machine1 systemd[1]: Started Network Name Resolution. container-test-run-monitoring> machine1 # [805055.641356] machine1 systemd[1]: Reached target Network. container-test-run-monitoring> machine1 # [805055.641393] machine1 systemd[1]: Reached target Network is Online. container-test-run-monitoring> machine1 # [805055.641422] machine1 systemd[1]: Reached target System Initialization. container-test-run-monitoring> machine1 # [805055.641459] machine1 systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container container-test-run-monitoring> machine1 # [805055.641477] machine1 systemd[1]: Started Daily Cleanup of Temporary Directories. container-test-run-monitoring> machine1 # [805055.641492] machine1 systemd[1]: Reached target Timer Units. container-test-run-monitoring> machine1 # [805055.641587] machine1 systemd[1]: Listening on D-Bus System Message Bus Socket. container-test-run-monitoring> machine1 # [805055.641719] machine1 systemd[1]: Listening on Nix Daemon Socket. container-test-run-monitoring> machine1 # [805055.641807] machine1 systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. container-test-run-monitoring> machine1 # [805055.641821] machine1 systemd[1]: Reached target Socket Units. container-test-run-monitoring> machine1 # [805055.641852] machine1 systemd[1]: Reached target Basic System. container-test-run-monitoring> machine1 # [805055.643508] machine1 systemd[1]: Started alloy.service. container-test-run-monitoring> machine1 # [805055.644684] machine1 systemd[1]: Starting Import lastlog data into lastlog2 database... container-test-run-monitoring> machine1 # [805055.645417] machine1 systemd[1]: Started Loki Service Daemon. container-test-run-monitoring> machine1 # [805055.646458] machine1 systemd[1]: Started mimir Service Daemon. container-test-run-monitoring> machine1 # [805055.647242] machine1 systemd[1]: Starting Nginx Web Server... container-test-run-monitoring> machine1 # [805055.672305] machine1 systemd[1]: Starting Name Service Cache Daemon (nsncd)... container-test-run-monitoring> machine1 # [805055.673229] machine1 systemd[1]: Starting PostgreSQL Server... container-test-run-monitoring> machine1 # [805055.794290] machine1 systemd[1]: lastlog2-import.service: Failed to spawn executor: No such file or directory container-test-run-monitoring> machine1 # [805055.794327] machine1 systemd[1]: lastlog2-import.service: Failed to spawn 'start-post' task: No such file or directory container-test-run-monitoring> machine1 # [805055.794359] machine1 systemd[1]: lastlog2-import.service: Failed with result 'resources'. container-test-run-monitoring> machine1 # [805055.794413] machine1 systemd[1]: Failed to start Import lastlog data into lastlog2 database. container-test-run-monitoring> machine1 # [805055.796151] machine1 systemd[1]: Starting D-Bus System Message Bus... container-test-run-monitoring> machine1 # [805056.009099] machine1 nsncd[252]: Sep 13 15:10:05.713 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" container-test-run-monitoring> machine1 # [805056.009200] machine1 systemd[1]: Started Name Service Cache Daemon (nsncd). container-test-run-monitoring> machine1 # [805056.009265] machine1 systemd[1]: Reached target Host and Network Name Lookups. container-test-run-monitoring> machine1 # [805056.009318] machine1 systemd[1]: Reached target User and Group Name Lookups. container-test-run-monitoring> machine1 # [805056.010228] machine1 systemd[1]: Starting User Login Management... container-test-run-monitoring> machine1 # [805056.010765] machine1 systemd[1]: Starting Permit User Sessions... container-test-run-monitoring> machine1 # [805056.018714] machine1 systemd[1]: Finished Permit User Sessions. container-test-run-monitoring> machine1 # [805056.019393] machine1 systemd[1]: Started Console Getty. container-test-run-monitoring> machine1 # [805056.019417] machine1 systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0 container-test-run-monitoring> machine1 # [805056.019428] machine1 systemd[1]: Reached target Login Prompts. container-test-run-monitoring> machine2 # [805055.976518] machine2 nsncd[229]: Sep 13 15:10:05.680 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" container-test-run-monitoring> machine2 # [805055.976505] machine2 systemd[1]: Started Name Service Cache Daemon (nsncd). container-test-run-monitoring> machine2 # [805055.976560] machine2 systemd[1]: Reached target Host and Network Name Lookups. container-test-run-monitoring> machine2 # [805055.976596] machine2 systemd[1]: Reached target User and Group Name Lookups. container-test-run-monitoring> machine2 # [805055.978173] machine2 systemd[1]: Starting User Login Management... container-test-run-monitoring> machine2 # [805055.978895] machine2 systemd[1]: Starting Permit User Sessions... container-test-run-monitoring> machine2 # [805056.008693] machine2 systemd[1]: Finished Permit User Sessions. container-test-run-monitoring> machine2 # [805056.009931] machine2 systemd[1]: Started Console Getty. container-test-run-monitoring> machine2 # [805056.009955] machine2 systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0 container-test-run-monitoring> machine2 # [805056.009967] machine2 systemd[1]: Reached target Login Prompts. container-test-run-monitoring> machine2 # [805056.187923] machine2 dbus-broker-launch[230]: Looking up NSS user entry for 'systemd-timesync'... container-test-run-monitoring> machine2 # [805056.188634] machine2 dbus-broker-launch[230]: NSS returned no entry for 'systemd-timesync' container-test-run-monitoring> machine2 # [805056.188634] machine2 dbus-broker-launch[230]: Invalid user-name in /nix/store/rl9kxhq4w21s81xsc4lyhbhnrnfqrnq9-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" container-test-run-monitoring> machine2 # [805056.189159] machine2 systemd[1]: Started D-Bus System Message Bus. container-test-run-monitoring> machine2 # [805056.193489] machine2 dbus-broker-launch[230]: Ready container-test-run-monitoring> machine1 # [805056.308552] machine1 dbus-broker-launch[264]: Looking up NSS user entry for 'systemd-timesync'... container-test-run-monitoring> machine1 # [805056.308966] machine1 dbus-broker-launch[264]: NSS returned no entry for 'systemd-timesync' container-test-run-monitoring> machine1 # [805056.308966] machine1 dbus-broker-launch[264]: Invalid user-name in /nix/store/zcj58v683cgwgbcd5x7sq8ny396673fi-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" container-test-run-monitoring> machine1 # [805056.309377] machine1 systemd[1]: Started D-Bus System Message Bus. container-test-run-monitoring> machine1 # [805056.314587] machine1 dbus-broker-launch[264]: Ready container-test-run-monitoring> machine1 # [805056.384536] machine1 mimir[250]: ts=2026-09-13T15:10:05.994022737Z caller=main.go:232 level=info msg="Starting application" version="(version=3.2.1, branch=unknown, revision=unknown)" container-test-run-monitoring> machine1 # [805056.384536] machine1 mimir[250]: ts=2026-09-13T15:10:05.996305276Z caller=server.go:401 level=info msg="server listening on addresses" http=[::]:3001 grpc=[::]:9095 container-test-run-monitoring> machine1 # [805056.384536] machine1 mimir[250]: ts=2026-09-13T15:10:06.016436186Z caller=memberlist_client.go:562 level=info component=memberlist msg="Using memberlist cluster label and node name" cluster_label= node=machine1-433fc6ad container-test-run-monitoring> machine1 # [805056.384536] machine1 mimir[250]: ts=2026-09-13T15:10:06.016701947Z caller=inmemory.go:137 level=info msg="created in-memory index cache" maxItemSizeBytes=134217728 maxSizeBytes=1073741824 maxItems=maxInt container-test-run-monitoring> machine1 # [805056.384536] machine1 mimir[250]: ts=2026-09-13T15:10:06.02262066Z caller=modules.go:1217 level=info msg="The ruler is not being started because you need to configure the ruler storage." container-test-run-monitoring> machine1 # [805056.384536] machine1 mimir[250]: ts=2026-09-13T15:10:06.024530438Z caller=ingester.go:414 level=warn msg="-blocks-storage.backend=filesystem is for development and testing only; you should switch to an external object store for production use or use a shared filesystem" container-test-run-monitoring> machine1 # [805056.384536] machine1 mimir[250]: ts=2026-09-13T15:10:06.025853393Z caller=ingester.go:584 level=info msg="TSDB idle compaction timeout set" timeout=1h4m52.758825229s container-test-run-monitoring> machine1 # [805056.384536] machine1 mimir[250]: ts=2026-09-13T15:10:06.026371535Z caller=modules.go:1240 level=info msg="The ruler storage has not been configured. Not starting the ruler." container-test-run-monitoring> machine1 # [805056.384536] machine1 mimir[250]: ts=2026-09-13T15:10:06.027200338Z caller=modules.go:1038 level=info msg="The query-frontend has not been configured with the query-scheduler address. Because Mimir is running in monolithic mode, it's attempting an automatic frontend configuration. If queries are unresponsive, consider explicitly configuring the query-scheduler address for querier-frontend." address=127.0.0.1:9095 container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.141791293Z caller=modules.go:755 level=info msg="The querier worker has not been configured with the query-scheduler address. Because Mimir is running in monolithic mode, it's attempting an automatic worker configuration. If queries are unresponsive, consider explicitly configuring the query-scheduler address for querier worker." address=127.0.0.1:9095 container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.141833144Z caller=worker.go:148 level=info msg="Starting querier worker connected to query-scheduler" scheduler=127.0.0.1:9095 container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.142765717Z caller=module_service.go:82 level=info msg=starting module=sanity-check container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.142750397Z caller=module_service.go:82 level=info msg=starting module=active-groups-cleanup-service container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.142765897Z caller=module_service.go:82 level=info msg=starting module=activity-tracker container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.143094749Z caller=sanity_check.go:34 level=info msg="Checking directories read/write access" container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.14355115Z caller=sanity_check.go:39 level=info msg="Directories read/write access successfully checked" container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.14355782Z caller=sanity_check.go:41 level=info msg="Checking object storage config" container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.144042672Z caller=sanity_check.go:46 level=info msg="Object storage config successfully checked" container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.144064802Z caller=module_service.go:82 level=info msg=starting module=server container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.144147613Z caller=module_service.go:82 level=info msg=starting module=memberlist-kv container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.144172393Z caller=module_service.go:82 level=info msg=starting module=ingester-ring container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.144238353Z caller=module_service.go:82 level=info msg=starting module=querier-ring container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.144229663Z caller=module_service.go:82 level=info msg=starting module=store-queryable container-test-run-monitoring> machine1 # [805056.484813] machine1 mimir[250]: ts=2026-09-13T15:10:06.144263723Z caller=module_service.go:82 level=info msg=starting module=compactor container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144241463Z caller=module_service.go:82 level=info msg=starting module=querier-lifecycler container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144251783Z caller=module_service.go:82 level=info msg=starting module=store-gateway container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144268773Z caller=module_service.go:82 level=info msg=starting module=query-scheduler container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144511374Z caller=ring.go:390 level=info msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144530374Z caller=ring.go:390 level=info msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144502764Z caller=ring.go:390 level=info msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144500764Z caller=ring.go:390 level=info msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144584044Z caller=module_service.go:82 level=info msg=starting module=ingester-service container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144586295Z caller=module_service.go:82 level=info msg=starting module=distributor-service container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144602195Z caller=module_service.go:82 level=info msg=starting module=query-frontend container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144767925Z caller=ring.go:390 level=info msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144833996Z caller=ingester_tsdb.go:415 level=info msg="opening existing TSDBs" container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144673835Z caller=basic_lifecycler.go:324 level=info msg="instance not found in the ring" instance=machine1 ring=distributor container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144899256Z caller=basic_lifecycler.go:324 level=info msg="instance not found in the ring" instance=machine1 ring=querier container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144893636Z caller=basic_lifecycler.go:324 level=info msg="instance not found in the ring" instance=machine1 ring=store-gateway container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144926296Z caller=basic_lifecycler_delegates.go:63 level=info msg="not loading tokens from file, tokens file path is empty" container-test-run-monitoring> machine1 # [805056.485123] machine1 mimir[250]: ts=2026-09-13T15:10:06.144859505Z caller=frontend_scheduler_worker.go:146 level=info msg="adding connection to query-scheduler" addr=127.0.0.1:9095 container-test-run-monitoring> machine1 # [805056.485360] machine1 mimir[250]: ts=2026-09-13T15:10:06.145189727Z caller=lifecycler.go:694 level=info msg="not loading tokens from file, tokens file path is empty" container-test-run-monitoring> machine1 # [805056.485360] machine1 mimir[250]: ts=2026-09-13T15:10:06.145238557Z caller=lifecycler.go:721 level=info msg="instance not found in ring, adding with no tokens" ring=ingester container-test-run-monitoring> machine1 # [805056.485360] machine1 mimir[250]: ts=2026-09-13T15:10:06.145375118Z caller=ring.go:390 level=info component=compactor msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [805056.485360] machine1 mimir[250]: ts=2026-09-13T15:10:06.145407338Z caller=basic_lifecycler.go:324 level=info component=compactor msg="instance not found in the ring" instance=machine1 ring=compactor container-test-run-monitoring> machine1 # [805056.485360] machine1 mimir[250]: ts=2026-09-13T15:10:06.145440868Z caller=lifecycler.go:562 level=info msg="auto-joining cluster after timeout" ring=ingester container-test-run-monitoring> machine1 # [805056.485360] machine1 mimir[250]: ts=2026-09-13T15:10:06.145276527Z caller=distributor.go:952 level=info msg="waiting until distributor is ACTIVE in the ring" container-test-run-monitoring> machine1 # [805056.485360] machine1 mimir[250]: ts=2026-09-13T15:10:06.145352298Z caller=gateway.go:290 level=info msg="waiting until store-gateway is JOINING in the ring" container-test-run-monitoring> machine1 # [805056.485360] machine1 mimir[250]: ts=2026-09-13T15:10:06.145521858Z caller=compactor.go:616 level=info component=compactor msg="waiting until compactor is ACTIVE in the ring" container-test-run-monitoring> machine1 # [805056.489046] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.095976871Z caller=main.go:146 msg="Starting Loki" version="(version=3.7.7, branch=unknown, revision=unknown)" container-test-run-monitoring> machine1 # [805056.489046] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.096009971Z caller=main.go:147 msg="Loading configuration file" filename=/nix/store/nd849x6dd4nlfkfl8nslkpw7lqapra25-validate-loki-conf container-test-run-monitoring> machine1 # [805056.489046] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.096628234Z caller=modules.go:1597 msg="Ruler storage is not configured; ruler will not be started." config_key=ruler.storage container-test-run-monitoring> machine1 # [805056.489046] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.112470297Z caller=server.go:397 msg="server listening on addresses" http=[::]:3002 grpc=[::]:9096 container-test-run-monitoring> machine1 # [805056.489046] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.132595227Z caller=table_manager.go:136 index-store=tsdb-2025-11-01 msg="uploading tables" container-test-run-monitoring> machine1 # [805056.489046] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.133681771Z caller=table_manager.go:300 index-store=tsdb-2025-11-01 msg="query readiness setup completed" duration=262.111µs distinct_users_len=0 distinct_users= container-test-run-monitoring> machine1 # [805056.489046] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.134126643Z caller=shipper.go:165 index-store=tsdb-2025-11-01 msg="starting index shipper in RW mode" container-test-run-monitoring> machine1 # [805056.489046] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.134648445Z caller=head_manager.go:313 index-store=tsdb-2025-11-01 component=tsdb-head-manager msg="loaded wals by period" groups=0 container-test-run-monitoring> machine1 # [805056.489046] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.134900516Z caller=manager.go:86 index-store=tsdb-2025-11-01 component=tsdb-manager msg="loaded leftover local indices" err=null successful=true buckets=0 indices=0 failures=0 container-test-run-monitoring> machine1 # [805056.489046] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.134924946Z caller=head_manager.go:313 index-store=tsdb-2025-11-01 component=tsdb-head-manager msg="loaded wals by period" groups=0 container-test-run-monitoring> machine1 # [805056.489046] machine1 loki[249]: level=warn ts=2026-09-13T15:10:06.144926196Z caller=modules.go:1643 msg="RulerStorage is nil. Not starting the ruler." container-test-run-monitoring> machine1 # [805056.489046] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.150846979Z caller=worker.go:134 component=querier msg="Starting querier worker using query-scheduler and scheduler ring for addresses" container-test-run-monitoring> machine1 # [805056.489046] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.164492874Z caller=module_service.go:82 msg=starting module=server container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.164503883Z caller=module_service.go:82 msg=starting module=cache-generation-loader container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.164769595Z caller=module_service.go:82 msg=starting module=memberlist-kv container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.164748995Z caller=module_service.go:82 msg=starting module=query-frontend-tripperware container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.164806755Z caller=module_service.go:82 msg=starting module=query-scheduler-ring container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.164858435Z caller=module_service.go:82 msg=starting module=store container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.164844605Z caller=module_service.go:82 msg=starting module=ring container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.165857479Z caller=ring.go:390 msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.165846539Z caller=basic_lifecycler.go:324 msg="instance not found in the ring" instance=machine1 ring=scheduler container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.165855159Z caller=ring.go:390 msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.165890539Z caller=basic_lifecycler_delegates.go:63 msg="not loading tokens from file, tokens file path is empty" container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.165964419Z caller=module_service.go:82 msg=starting module=ingester-querier container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.165974669Z caller=module_service.go:82 msg=starting module=compactor container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.166015469Z caller=module_service.go:82 msg=starting module=distributor container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.16602598Z caller=module_service.go:82 msg=starting module=rule-evaluator container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.16603149Z caller=module_service.go:82 msg=starting module=ingester container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.16609478Z caller=basic_lifecycler.go:324 component=distributor msg="instance not found in the ring" instance=machine1 ring=distributor container-test-run-monitoring> machine1 # [805056.489513] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.16612084Z caller=ring.go:390 component=distributor msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.16615915Z caller=ingester.go:571 component=ingester msg="recovering from checkpoint" container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.166348611Z caller=ring.go:390 msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.166372871Z caller=basic_lifecycler.go:324 msg="instance not found in the ring" instance=machine1 ring=compactor container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.166384511Z caller=basic_lifecycler_delegates.go:63 msg="not loading tokens from file, tokens file path is empty" container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.166397381Z caller=ringmanager.go:186 msg="waiting until scheduler is JOINING in the ring" container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.166513422Z caller=compactor.go:420 msg="waiting until compactor is JOINING in the ring" container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.166501462Z caller=checkpoint.go:549 msg="tmp checkpoint cleanup starting" container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.166691462Z caller=checkpoint.go:554 msg="tmp checkpoint cleanup done" duration=46.85µs success=true container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=error ts=2026-09-13T15:10:06.166444761Z caller=ratestore.go:110 msg="error getting ingester clients" err="empty ring" container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.166726082Z caller=recovery.go:45 component=ingester msg="no checkpoint found, treating as no-op" container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.167570496Z caller=ingester.go:587 component=ingester msg="recovered WAL checkpoint recovery finished" elapsed=1.440026ms errors=false container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.167588706Z caller=ingester.go:593 component=ingester msg="recovering from WAL" container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.16859931Z caller=ingester.go:609 component=ingester msg="WAL segment recovery finished" elapsed=2.45707ms errors=false container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.16865649Z caller=ingester.go:557 component=ingester msg="closing recoverer" container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.16867657Z caller=ingester.go:565 component=ingester msg="WAL recovery finished" time=2.54677ms container-test-run-monitoring> machine1 # [805056.489720] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.168808661Z caller=wal.go:173 msg=started component=wal container-test-run-monitoring> machine1 # [805056.489914] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.168853521Z caller=lifecycler.go:694 component=ingester msg="not loading tokens from file, tokens file path is empty" container-test-run-monitoring> machine1 # [805056.489914] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.168816721Z caller=ingester.go:778 component=ingester msg="sleeping for initial delay before starting periodic flushing" delay=18.168614573s container-test-run-monitoring> machine1 # [805056.489914] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.168947921Z caller=lifecycler.go:721 component=ingester msg="instance not found in ring, adding with no tokens" ring=ingester container-test-run-monitoring> machine1 # [805056.489914] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.169038112Z caller=lifecycler.go:562 component=ingester msg="auto-joining cluster after timeout" ring=ingester container-test-run-monitoring> machine1 # [805056.684807] machine1 mimir[250]: ts=2026-09-13T15:10:06.310678944Z caller=compactor.go:621 level=info component=compactor msg="compactor is ACTIVE in the ring" container-test-run-monitoring> machine1 # [805056.684807] machine1 mimir[250]: ts=2026-09-13T15:10:06.310865204Z caller=compactor.go:743 level=info component=compactor msg="discovering users from bucket" container-test-run-monitoring> machine1 # [805056.684807] machine1 mimir[250]: ts=2026-09-13T15:10:06.310929385Z caller=compactor.go:753 level=info component=compactor msg="discovered users from bucket" users=0 container-test-run-monitoring> machine1 # [805056.684807] machine1 mimir[250]: ts=2026-09-13T15:10:06.310982815Z caller=blocks_cleaner.go:244 level=info component=cleaner task=clean_up_users_during_startup msg="started blocks cleanup and maintenance" container-test-run-monitoring> machine1 # [805056.684807] machine1 mimir[250]: ts=2026-09-13T15:10:06.311828418Z caller=blocks_cleaner.go:250 level=info component=cleaner task=clean_up_users_during_startup msg="successfully completed blocks cleanup and maintenance" container-test-run-monitoring> machine1 # [805056.684807] machine1 mimir[250]: ts=2026-09-13T15:10:06.32481836Z caller=gateway.go:294 level=info msg="store-gateway is JOINING in the ring" container-test-run-monitoring> machine1 # [805056.684807] machine1 mimir[250]: ts=2026-09-13T15:10:06.32488361Z caller=bucket_stores.go:231 level=info msg="synchronizing TSDB blocks for all users" container-test-run-monitoring> machine1 # [805056.684807] machine1 mimir[250]: ts=2026-09-13T15:10:06.325101861Z caller=bucket_stores.go:244 level=info msg="successfully synchronized TSDB blocks for all users" container-test-run-monitoring> machine1 # [805056.684807] machine1 mimir[250]: ts=2026-09-13T15:10:06.342984722Z caller=module_service.go:82 level=info msg=starting module=querier container-test-run-monitoring> machine1 # [805056.684807] machine1 mimir[250]: ts=2026-09-13T15:10:06.343098503Z caller=worker.go:248 level=info msg="adding connection" addr=127.0.0.1:9095 in-use=true container-test-run-monitoring> machine1 # [805056.688926] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.33497504Z caller=ringmanager.go:190 msg="scheduler is JOINING in the ring" container-test-run-monitoring> machine1 # [805056.688926] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.335066731Z caller=ringmanager.go:199 msg="waiting until scheduler is ACTIVE in the ring" container-test-run-monitoring> machine1 # [805056.688926] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.354065546Z caller=compactor.go:424 msg="compactor is JOINING in the ring" container-test-run-monitoring> machine1 # [805056.688926] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.354192376Z caller=compactor.go:434 msg="waiting until compactor is ACTIVE in the ring" container-test-run-monitoring> machine1 # [805056.759810] machine1 alloy[247]: ts=2026-09-13T15:10:06.444265724Z level=info msg="Alloy is starting" container-test-run-monitoring> machine1 # [805056.759810] machine1 alloy[247]: ts=2026-09-13T15:10:06.444503095Z level=info msg="boringcrypto enabled" enabled=false container-test-run-monitoring> machine1 # [805056.759810] machine1 alloy[247]: ts=2026-09-13T15:10:06.451075931Z level=info msg="memory is not limited, skipping" package=github.com/KimMachineGun/automemlimit/memlimit container-test-run-monitoring> machine1 # [805056.759810] machine1 alloy[247]: ts=2026-09-13T15:10:06.451155121Z level=info msg="no peer discovery configured: both join and discover peers are empty" service=cluster container-test-run-monitoring> machine1 # [805056.759810] machine1 alloy[247]: ts=2026-09-13T15:10:06.454790316Z level=info msg="starting complete graph evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 container-test-run-monitoring> machine1 # [805056.759810] machine1 alloy[247]: ts=2026-09-13T15:10:06.454874916Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=labelstore duration=59.29µs container-test-run-monitoring> machine1 # [805056.760308] machine1 alloy[247]: ts=2026-09-13T15:10:06.456703323Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=loki.write.loki duration=1.825217ms container-test-run-monitoring> machine1 # [805056.760308] machine1 alloy[247]: ts=2026-09-13T15:10:06.458104469Z level=info msg="replaying WAL, this may take a while" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=wal dir=data-alloy/prometheus.remote_write.mimir/wal container-test-run-monitoring> machine1 # [805056.760308] machine1 alloy[247]: ts=2026-09-13T15:10:06.45840316Z level=info msg="WAL segment loaded" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=wal segment=0 maxSegment=0 container-test-run-monitoring> machine1 # [805056.760308] machine1 alloy[247]: ts=2026-09-13T15:10:06.459599585Z level=info msg="Starting WAL watcher" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=rw remote_name=8a3c26 url=http://machine1.clan/mimir/api/v1/push queue=8a3c26 container-test-run-monitoring> machine1 # [805056.760308] machine1 alloy[247]: ts=2026-09-13T15:10:06.459641615Z level=info msg="Starting scraped metadata watcher" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=rw remote_name=8a3c26 url=http://machine1.clan/mimir/api/v1/push container-test-run-monitoring> machine1 # [805056.760308] machine1 alloy[247]: ts=2026-09-13T15:10:06.459659025Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=prometheus.remote_write.mimir duration=2.951652ms container-test-run-monitoring> machine1 # [805056.760308] machine1 alloy[247]: ts=2026-09-13T15:10:06.460097787Z level=info msg="Replaying WAL" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=rw remote_name=8a3c26 url=http://machine1.clan/mimir/api/v1/push queue=8a3c26 container-test-run-monitoring> machine1 # [805056.760308] machine1 alloy[247]: ts=2026-09-13T15:10:06.460344248Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=prometheus.relabel.create_nixos_services_metric duration=680.593µs container-test-run-monitoring> machine1 # [805056.760308] machine1 alloy[247]: ts=2026-09-13T15:10:06.460353668Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=tracing duration=6.11µs container-test-run-monitoring> machine1 # [805056.760565] machine1 alloy[247]: ts=2026-09-13T15:10:06.460387278Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=remotecfg duration=31.8µs container-test-run-monitoring> machine1 # [805056.760565] machine1 alloy[247]: ts=2026-09-13T15:10:06.460401168Z level=info msg="applying non-TLS config to HTTP server" service=http container-test-run-monitoring> machine1 # [805056.760565] machine1 alloy[247]: ts=2026-09-13T15:10:06.460402178Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=http duration=13.12µs container-test-run-monitoring> machine1 # [805056.760565] machine1 alloy[247]: ts=2026-09-13T15:10:06.460406818Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=cluster duration=2.05µs container-test-run-monitoring> machine1 # [805056.760565] machine1 alloy[247]: ts=2026-09-13T15:10:06.460412498Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=otel duration=4.03µs container-test-run-monitoring> machine1 # [805056.760565] machine1 alloy[247]: ts=2026-09-13T15:10:06.460559249Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=loki.relabel.journal duration=145.351µs container-test-run-monitoring> machine1 # [805056.760565] machine1 alloy[247]: ts=2026-09-13T15:10:06.46075762Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=loki.source.journal.all duration=195.621µs container-test-run-monitoring> machine1 # [805056.760565] machine1 alloy[247]: ts=2026-09-13T15:10:06.462226345Z level=info msg="Parsed flag --collector.systemd.unit-include" component_path=/ component_id=prometheus.exporter.unix.local_system collector=systemd flag=.+ container-test-run-monitoring> machine1 # [805056.760565] machine1 alloy[247]: ts=2026-09-13T15:10:06.462235785Z level=info msg="Parsed flag --collector.systemd.unit-exclude" component_path=/ component_id=prometheus.exporter.unix.local_system collector=systemd flag=.+\.(automount|device|mount|scope|slice) container-test-run-monitoring> machine1 # [805056.760565] machine1 alloy[247]: ts=2026-09-13T15:10:06.462258856Z level=info msg="Parsed flag --collector.filesystem.mount-points-exclude" component_path=/ component_id=prometheus.exporter.unix.local_system collector=filesystem flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) container-test-run-monitoring> machine1 # [805056.760770] machine1 alloy[247]: ts=2026-09-13T15:10:06.462259776Z level=info msg="Parsed flag --collector.filesystem.fs-types-exclude" component_path=/ component_id=prometheus.exporter.unix.local_system collector=filesystem flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ container-test-run-monitoring> machine1 # [805056.760770] machine1 alloy[247]: ts=2026-09-13T15:10:06.462333936Z level=info msg="Enabled node_exporter collectors" component_path=/ component_id=prometheus.exporter.unix.local_system container-test-run-monitoring> machine1 # [805056.760770] machine1 alloy[247]: ts=2026-09-13T15:10:06.462335236Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=cpu container-test-run-monitoring> machine1 # [805056.760770] machine1 alloy[247]: ts=2026-09-13T15:10:06.462342846Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=filesystem container-test-run-monitoring> machine1 # [805056.760770] machine1 alloy[247]: ts=2026-09-13T15:10:06.462343406Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=meminfo container-test-run-monitoring> machine1 # [805056.760770] machine1 alloy[247]: ts=2026-09-13T15:10:06.462343876Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=systemd container-test-run-monitoring> machine1 # [805056.760770] machine1 alloy[247]: ts=2026-09-13T15:10:06.462371466Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=prometheus.exporter.unix.local_system duration=1.611076ms container-test-run-monitoring> machine1 # [805056.760770] machine1 alloy[247]: ts=2026-09-13T15:10:06.464043093Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=prometheus.scrape.scrape_metrics duration=1.668107ms container-test-run-monitoring> machine1 # [805056.760770] machine1 alloy[247]: ts=2026-09-13T15:10:06.464212273Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=logging duration=165.4µs container-test-run-monitoring> machine1 # [805056.760770] machine1 alloy[247]: ts=2026-09-13T15:10:06.464224113Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=livedebugging duration=6.65µs container-test-run-monitoring> machine1 # [805056.760980] machine1 alloy[247]: ts=2026-09-13T15:10:06.464231013Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 node_id=ui duration=2.48µs container-test-run-monitoring> machine1 # [805056.760980] machine1 alloy[247]: ts=2026-09-13T15:10:06.464234623Z level=info msg="finished complete graph evaluation" controller_path=/ controller_id="" trace_id=335679e4ffe8f29adc155c94b5a51239 duration=9.849949ms container-test-run-monitoring> machine1 # [805056.760980] machine1 alloy[247]: ts=2026-09-13T15:10:06.464338724Z level=info msg="scheduling loaded components and services" controller_id="" container-test-run-monitoring> machine1 # [805056.760980] machine1 alloy[247]: ts=2026-09-13T15:10:06.464385024Z level=info msg="{^_^} Alloy is running" container-test-run-monitoring> machine1 # [805056.760980] machine1 alloy[247]: ts=2026-09-13T15:10:06.464630915Z level=info msg="starting cluster node" service=cluster peers_count=0 peers="" advertise_addr=127.0.0.1:12345 minimum_cluster_size=0 minimum_size_wait_timeout=0s container-test-run-monitoring> machine1 # [805056.760980] machine1 alloy[247]: ts=2026-09-13T15:10:06.464751706Z level=error msg="failed to register collector with remote server" service=remotecfg id=e289d5a8-e8ba-45f3-ae69-2b4ce2bf8644 name="" err="noop client" container-test-run-monitoring> machine1 # [805056.760980] machine1 alloy[247]: ts=2026-09-13T15:10:06.464896016Z level=info msg="peers changed" service=cluster peers_count=1 min_cluster_size=0 peers=machine1 container-test-run-monitoring> machine1 # [805056.761134] machine1 alloy[247]: ts=2026-09-13T15:10:06.465325888Z level=info msg="now listening for http traffic" service=http addr=127.0.0.1:12345 container-test-run-monitoring> machine1 # [805056.841762] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.518637529Z caller=ringmanager.go:203 msg="scheduler is ACTIVE in the ring" container-test-run-monitoring> machine1 # [805056.841762] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.51868902Z caller=module_service.go:82 msg=starting module=query-scheduler container-test-run-monitoring> machine1 # [805056.841762] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.519121831Z caller=module_service.go:82 msg=starting module=querier container-test-run-monitoring> machine1 # [805056.841762] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.519135231Z caller=module_service.go:82 msg=starting module=query-frontend container-test-run-monitoring> machine1 # [805056.841762] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.545938838Z caller=compactor.go:438 msg="compactor is ACTIVE in the ring" container-test-run-monitoring> machine1 # [805056.841762] machine1 loki[249]: level=info ts=2026-09-13T15:10:06.546002598Z caller=loki.go:615 msg="Loki started" startup_time=507.195474ms container-test-run-monitoring> machine1 # [805056.870613] machine1 nginx-pre-start[398]: nginx: the configuration file /nix/store/xn1w4yb4zxxsi2dbbirrx8jmsv6bfcs5-nginx.conf syntax is ok container-test-run-monitoring> machine1 # [805056.870976] machine1 nginx-pre-start[398]: nginx: configuration file /nix/store/xn1w4yb4zxxsi2dbbirrx8jmsv6bfcs5-nginx.conf test is successful container-test-run-monitoring> machine1 # [805056.885162] machine1 systemd-networkd[234]: eth1: Gained IPv6LL container-test-run-monitoring> machine1 # [805056.943238] machine1 postgresql-pre-start[405]: The files belonging to this database system will be owned by user "postgres". container-test-run-monitoring> machine1 # [805056.943238] machine1 postgresql-pre-start[405]: This user must also own the server process. container-test-run-monitoring> machine1 # [805056.943557] machine1 postgresql-pre-start[405]: The database cluster will be initialized with locale "en_US.UTF-8". container-test-run-monitoring> machine1 # [805056.943557] machine1 postgresql-pre-start[405]: The default database encoding has accordingly been set to "UTF8". container-test-run-monitoring> machine1 # [805056.943557] machine1 postgresql-pre-start[405]: The default text search configuration will be set to "english". container-test-run-monitoring> machine1 # [805056.943557] machine1 postgresql-pre-start[405]: Data page checksums are enabled. container-test-run-monitoring> machine1 # [805056.943557] machine1 postgresql-pre-start[405]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok container-test-run-monitoring> machine2 # [805056.755837] machine2 alloy[227]: ts=2026-09-13T15:10:06.444268104Z level=info msg="Alloy is starting" container-test-run-monitoring> machine2 # [805056.755837] machine2 alloy[227]: ts=2026-09-13T15:10:06.444500975Z level=info msg="boringcrypto enabled" enabled=false container-test-run-monitoring> machine2 # [805056.755837] machine2 alloy[227]: ts=2026-09-13T15:10:06.451778424Z level=info msg="memory is not limited, skipping" package=github.com/KimMachineGun/automemlimit/memlimit container-test-run-monitoring> machine2 # [805056.756319] machine2 alloy[227]: ts=2026-09-13T15:10:06.451804524Z level=info msg="no peer discovery configured: both join and discover peers are empty" service=cluster container-test-run-monitoring> machine2 # [805056.756319] machine2 alloy[227]: ts=2026-09-13T15:10:06.454792406Z level=info msg="starting complete graph evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 container-test-run-monitoring> machine2 # [805056.756319] machine2 alloy[227]: ts=2026-09-13T15:10:06.455962851Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=loki.relabel.journal duration=1.149955ms container-test-run-monitoring> machine2 # [805056.756319] machine2 alloy[227]: ts=2026-09-13T15:10:06.457603027Z level=info msg="replaying WAL, this may take a while" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=wal dir=data-alloy/prometheus.remote_write.mimir/wal container-test-run-monitoring> machine2 # [805056.756319] machine2 alloy[227]: ts=2026-09-13T15:10:06.45840212Z level=info msg="WAL segment loaded" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=wal segment=0 maxSegment=0 container-test-run-monitoring> machine2 # [805056.756319] machine2 alloy[227]: ts=2026-09-13T15:10:06.459617155Z level=info msg="Starting WAL watcher" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=rw remote_name=740266 url=http://machine1.clan/mimir/api/v1/push queue=740266 container-test-run-monitoring> machine2 # [805056.756319] machine2 alloy[227]: ts=2026-09-13T15:10:06.459638885Z level=info msg="Starting scraped metadata watcher" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=rw remote_name=740266 url=http://machine1.clan/mimir/api/v1/push container-test-run-monitoring> machine2 # [805056.756319] machine2 alloy[227]: ts=2026-09-13T15:10:06.459656755Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=prometheus.remote_write.mimir duration=3.690484ms container-test-run-monitoring> machine2 # [805056.756319] machine2 alloy[227]: ts=2026-09-13T15:10:06.460007057Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=prometheus.relabel.create_nixos_services_metric duration=345.652µs container-test-run-monitoring> machine2 # [805056.756319] machine2 alloy[227]: ts=2026-09-13T15:10:06.460188617Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=logging duration=178.6µs container-test-run-monitoring> machine2 # [805056.756522] machine2 alloy[227]: ts=2026-09-13T15:10:06.460267448Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=remotecfg duration=74.261µs container-test-run-monitoring> machine2 # [805056.756522] machine2 alloy[227]: ts=2026-09-13T15:10:06.460327098Z level=info msg="applying non-TLS config to HTTP server" service=http container-test-run-monitoring> machine2 # [805056.756522] machine2 alloy[227]: ts=2026-09-13T15:10:06.460333628Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=http duration=60.2µs container-test-run-monitoring> machine2 # [805056.756522] machine2 alloy[227]: ts=2026-09-13T15:10:06.460339888Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=cluster duration=2.02µs container-test-run-monitoring> machine2 # [805056.756522] machine2 alloy[227]: ts=2026-09-13T15:10:06.460345568Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=otel duration=1.7µs container-test-run-monitoring> machine2 # [805056.756522] machine2 alloy[227]: ts=2026-09-13T15:10:06.460354238Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=livedebugging duration=4.49µs container-test-run-monitoring> machine2 # [805056.756522] machine2 alloy[227]: ts=2026-09-13T15:10:06.460359548Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=ui duration=1.2µs container-test-run-monitoring> machine2 # [805056.756522] machine2 alloy[227]: ts=2026-09-13T15:10:06.460365818Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=labelstore duration=2.45µs container-test-run-monitoring> machine2 # [805056.756522] machine2 alloy[227]: ts=2026-09-13T15:10:06.460092947Z level=info msg="Replaying WAL" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=rw remote_name=740266 url=http://machine1.clan/mimir/api/v1/push queue=740266 container-test-run-monitoring> machine2 # [805056.756908] machine2 alloy[227]: ts=2026-09-13T15:10:06.461188341Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=loki.write.loki duration=817.473µs container-test-run-monitoring> machine2 # [805056.757092] machine2 alloy[227]: ts=2026-09-13T15:10:06.461377912Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=loki.source.journal.all duration=173.681µs container-test-run-monitoring> machine2 # [805056.757952] machine2 alloy[227]: ts=2026-09-13T15:10:06.462233345Z level=info msg="Parsed flag --collector.filesystem.mount-points-exclude" component_path=/ component_id=prometheus.exporter.unix.local_system collector=filesystem flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) container-test-run-monitoring> machine2 # [805056.757969] machine2 alloy[227]: ts=2026-09-13T15:10:06.462247016Z level=info msg="Parsed flag --collector.filesystem.fs-types-exclude" component_path=/ component_id=prometheus.exporter.unix.local_system collector=filesystem flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ container-test-run-monitoring> machine2 # [805056.758152] machine2 alloy[227]: ts=2026-09-13T15:10:06.462436406Z level=info msg="Parsed flag --collector.systemd.unit-include" component_path=/ component_id=prometheus.exporter.unix.local_system collector=systemd flag=.+ container-test-run-monitoring> machine2 # [805056.758171] machine2 alloy[227]: ts=2026-09-13T15:10:06.462451746Z level=info msg="Parsed flag --collector.systemd.unit-exclude" component_path=/ component_id=prometheus.exporter.unix.local_system collector=systemd flag=.+\.(automount|device|mount|scope|slice) container-test-run-monitoring> machine2 # [805056.758247] machine2 alloy[227]: ts=2026-09-13T15:10:06.462533277Z level=info msg="Enabled node_exporter collectors" component_path=/ component_id=prometheus.exporter.unix.local_system container-test-run-monitoring> machine2 # [805056.758273] machine2 alloy[227]: ts=2026-09-13T15:10:06.462542347Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=cpu container-test-run-monitoring> machine2 # [805056.758273] machine2 alloy[227]: ts=2026-09-13T15:10:06.462548237Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=filesystem container-test-run-monitoring> machine2 # [805056.758273] machine2 alloy[227]: ts=2026-09-13T15:10:06.462551917Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=meminfo container-test-run-monitoring> machine2 # [805056.758273] machine2 alloy[227]: ts=2026-09-13T15:10:06.462555257Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=systemd container-test-run-monitoring> machine2 # [805056.758319] machine2 alloy[227]: ts=2026-09-13T15:10:06.462568957Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=prometheus.exporter.unix.local_system duration=1.179725ms container-test-run-monitoring> machine2 # [805056.759732] machine2 alloy[227]: ts=2026-09-13T15:10:06.464015972Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=prometheus.scrape.scrape_metrics duration=1.437376ms container-test-run-monitoring> machine2 # [805056.759764] machine2 alloy[227]: ts=2026-09-13T15:10:06.464033952Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 node_id=tracing duration=6.3µs container-test-run-monitoring> machine2 # [805056.759764] machine2 alloy[227]: ts=2026-09-13T15:10:06.464038763Z level=info msg="finished complete graph evaluation" controller_path=/ controller_id="" trace_id=c36710af14eb4ff2bd35991900f45768 duration=9.650799ms container-test-run-monitoring> machine2 # [805056.759956] machine2 alloy[227]: ts=2026-09-13T15:10:06.464204563Z level=info msg="scheduling loaded components and services" controller_id="" container-test-run-monitoring> machine2 # [805056.759956] machine2 alloy[227]: ts=2026-09-13T15:10:06.464248593Z level=info msg="{^_^} Alloy is running" container-test-run-monitoring> machine2 # [805056.760443] machine2 alloy[227]: ts=2026-09-13T15:10:06.464637475Z level=info msg="starting cluster node" service=cluster peers_count=0 peers="" advertise_addr=127.0.0.1:12345 minimum_cluster_size=0 minimum_size_wait_timeout=0s container-test-run-monitoring> machine2 # [805056.760574] machine2 alloy[227]: ts=2026-09-13T15:10:06.464779795Z level=error msg="failed to register collector with remote server" service=remotecfg id=fbc7bf55-a616-437d-95c4-5b7d443cb2b2 name="" err="noop client" container-test-run-monitoring> machine2 # [805056.760619] machine2 alloy[227]: ts=2026-09-13T15:10:06.464889566Z level=info msg="peers changed" service=cluster peers_count=1 min_cluster_size=0 peers=machine2 container-test-run-monitoring> machine2 # [805056.761077] machine2 alloy[227]: ts=2026-09-13T15:10:06.465349778Z level=info msg="now listening for http traffic" service=http addr=127.0.0.1:12345 container-test-run-monitoring> machine2 # [805056.885095] machine2 systemd-networkd[214]: eth1: Gained IPv6LL container-test-run-monitoring> machine2 # [805057.033728] machine2 systemd-logind[246]: New seat seat0. container-test-run-monitoring> machine2 # [805057.033904] machine2 systemd[1]: Started User Login Management. container-test-run-monitoring> machine1 # [805056.944346] machine1 postgresql-pre-start[405]: creating subdirectories ... ok container-test-run-monitoring> machine1 # [805056.944457] machine1 postgresql-pre-start[405]: selecting dynamic shared memory implementation ... posix container-test-run-monitoring> machine1 # [805056.962923] machine1 postgresql-pre-start[405]: selecting default "max_connections" ... 100 container-test-run-monitoring> machine1 # [805056.985868] machine1 postgresql-pre-start[405]: selecting default "shared_buffers" ... 128MB container-test-run-monitoring> machine1 # [805057.049808] machine1 systemd-logind[275]: New seat seat0. container-test-run-monitoring> machine1 # [805057.049951] machine1 systemd[1]: Started User Login Management. container-test-run-monitoring> machine1 # [805057.059333] machine1 systemd[1]: Starting linger-users.service... container-test-run-monitoring> machine1 # [805057.069426] machine1 systemd[1]: linger-users.service: Deactivated successfully. container-test-run-monitoring> machine1 # [805057.069479] machine1 systemd[1]: Finished linger-users.service. container-test-run-monitoring> machine2 # [805057.034757] machine2 systemd[1]: Starting linger-users.service... container-test-run-monitoring> machine2 # [805057.066377] machine2 systemd[1]: linger-users.service: Deactivated successfully. container-test-run-monitoring> machine2 # [805057.066689] machine2 systemd[1]: Finished linger-users.service. container-test-run-monitoring> machine2 # [805057.068528] machine2 systemd[1]: Reached target Multi-User System. container-test-run-monitoring> machine2 # [805057.068710] machine2 systemd[1]: Startup finished in 3.814s. container-test-run-monitoring> machine1 # [805057.223750] machine1 postgresql-pre-start[405]: selecting default time zone ... UTC container-test-run-monitoring> machine1 # [805057.224492] machine1 postgresql-pre-start[405]: creating configuration files ... ok container-test-run-monitoring> machine1 # [805057.349251] machine1 postgresql-pre-start[405]: running bootstrap script ... ok container-test-run-monitoring> machine1 # [805057.655472] machine1 postgresql-pre-start[405]: performing post-bootstrap initialization ... ok container-test-run-monitoring> machine1 # [805057.684728] machine1 mimir[250]: ts=2026-09-13T15:10:07.325660913Z caller=gateway.go:330 level=info msg="waiting until store-gateway is ACTIVE in the ring" container-test-run-monitoring> machine1 # [805057.684728] machine1 mimir[250]: ts=2026-09-13T15:10:07.325703293Z caller=gateway.go:334 level=info msg="store-gateway is ACTIVE in the ring" container-test-run-monitoring> machine1 # [805057.684728] machine1 mimir[250]: ts=2026-09-13T15:10:07.325731643Z caller=mimir.go:1170 level=info msg="Application started" container-test-run-monitoring> machine1 # [805057.850018] machine1 postgresql-pre-start[405]: syncing data to disk ... ok container-test-run-monitoring> machine1 # [805057.850018] machine1 postgresql-pre-start[405]: initdb: warning: enabling "trust" authentication for local connections container-test-run-monitoring> machine1 # [805057.850018] machine1 postgresql-pre-start[405]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. container-test-run-monitoring> machine1 # [805057.850018] machine1 postgresql-pre-start[405]: Success. You can now start the database server using: container-test-run-monitoring> machine1 # [805057.850018] machine1 postgresql-pre-start[405]: pg_ctl -D /var/lib/postgresql/18 -l logfile start container-test-run-monitoring> machine1 # [805058.048159] machine1 systemd[1]: Started Nginx Web Server. container-test-run-monitoring> machine1 # [805059.006512] machine1 postgres[421]: [421] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit container-test-run-monitoring> machine1 # [805059.007795] machine1 postgres[421]: [421] LOG: listening on IPv6 address "::1", port 5432 container-test-run-monitoring> machine1 # [805059.007795] machine1 postgres[421]: [421] LOG: listening on IPv4 address "127.0.0.1", port 5432 container-test-run-monitoring> machine1 # [805059.008897] machine1 postgres[421]: [421] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" container-test-run-monitoring> machine1 # [805059.011808] machine1 postgres[437]: [437] LOG: database system was shut down at 2026-09-13 15:10:07 GMT container-test-run-monitoring> machine1 # [805059.014831] machine1 postgres[421]: [421] LOG: database system is ready to accept connections container-test-run-monitoring> machine1 # [805059.888346] machine1 loki[249]: level=info ts=2026-09-13T15:10:09.520003064Z caller=scheduler.go:658 msg="this scheduler is in the ReplicationSet, will now accept requests." container-test-run-monitoring> machine1 # [805059.888346] machine1 loki[249]: level=info ts=2026-09-13T15:10:09.520302785Z caller=worker.go:235 component=querier msg="adding connection" addr=127.0.0.1:9096 container-test-run-monitoring> machine1 # [805060.124158] machine1 postgresql-post-start[450]: CREATE ROLE container-test-run-monitoring> machine1 # [805060.181341] machine1 postgresql-post-start[455]: CREATE DATABASE container-test-run-monitoring> machine1 # [805060.185380] machine1 systemd[1]: Started PostgreSQL Server. container-test-run-monitoring> machine1 # [805060.186316] machine1 systemd[1]: Starting PostgreSQL Setup Scripts... container-test-run-monitoring> machine1 # [805060.219401] machine1 systemd[1]: Finished PostgreSQL Setup Scripts. container-test-run-monitoring> machine1 # [805060.219649] machine1 systemd[1]: Reached target PostgreSQL. container-test-run-monitoring> machine1 # [805060.220483] machine1 systemd[1]: Starting Grafana Service Daemon... container-test-run-monitoring> machine1 # [805061.888891] machine1 loki[249]: level=info ts=2026-09-13T15:10:11.547125851Z caller=compactor.go:497 msg="this instance has been chosen to run the compactor, starting compactor" container-test-run-monitoring> machine1 # [805061.888891] machine1 loki[249]: level=info ts=2026-09-13T15:10:11.547250911Z caller=tables_manager.go:70 msg="waiting 10m0s for ring to stay stable and previous compactions to finish before starting compactor" container-test-run-monitoring> machine1 # [805062.333948] machine1 alloy[247]: ts=2026-09-13T15:10:12.03817743Z level=info msg="Done replaying WAL" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=rw remote_name=8a3c26 url=http://machine1.clan/mimir/api/v1/push duration=5.578499355s container-test-run-monitoring> machine1 # [805062.384459] machine1 mimir[250]: ts=2026-09-13T15:10:12.075267058Z caller=handler.go:95 level=info user=anonymous caller=head.go:776 time=2026-09-13T15:10:12.074450734Z msg="Replaying on-disk memory mappable chunks if any" container-test-run-monitoring> machine1 # [805062.384459] machine1 mimir[250]: ts=2026-09-13T15:10:12.075416918Z caller=handler.go:95 level=info user=anonymous caller=head.go:862 time=2026-09-13T15:10:12.075379748Z msg="On-disk memory mappable chunks replay completed" duration=23.62µs container-test-run-monitoring> machine1 # [805062.384459] machine1 mimir[250]: ts=2026-09-13T15:10:12.075441758Z caller=handler.go:95 level=info user=anonymous caller=head.go:870 time=2026-09-13T15:10:12.075438818Z msg="Replaying WAL, this may take a while" container-test-run-monitoring> machine1 # [805062.384459] machine1 mimir[250]: ts=2026-09-13T15:10:12.079659855Z caller=handler.go:95 level=info user=anonymous caller=head.go:966 time=2026-09-13T15:10:12.079616985Z msg="WAL segment loaded" segment=0 maxSegment=0 duration=2.769601ms container-test-run-monitoring> machine1 # [805062.384459] machine1 mimir[250]: ts=2026-09-13T15:10:12.079696965Z caller=handler.go:95 level=info user=anonymous caller=head.go:1003 time=2026-09-13T15:10:12.079688855Z msg="WAL replay completed" checkpoint_replay_duration=1.398116ms wal_replay_duration=2.842501ms wbl_replay_duration=50ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=23.62µs total_replay_duration=4.341307ms container-test-run-monitoring> machine1 # [805062.384459] machine1 mimir[250]: ts=2026-09-13T15:10:12.082651077Z caller=handler.go:95 level=info user=anonymous caller=db.go:2729 time=2026-09-13T15:10:12.082629187Z msg="Compactions disabled" container-test-run-monitoring> machine1 # [805062.384459] machine1 mimir[250]: ts=2026-09-13T15:10:12.082678087Z caller=ingester_tsdb.go:252 level=info user=anonymous msg="Running compaction after WAL replay" container-test-run-monitoring> machine1: still waiting for container 'machine1' to reach ready state... container-test-run-monitoring> machine1 # [805062.974631] machine1 systemd[1]: Started Grafana Service Daemon. container-test-run-monitoring> machine1 # [805062.974835] machine1 systemd[1]: Reached target Multi-User System. container-test-run-monitoring> machine1 # [805062.975215] machine1 systemd[1]: Startup finished in 9.696s. container-test-run-monitoring> machine1: (finished: waiting for unit multi-user.target, in 10.65 seconds) container-test-run-monitoring> machine2: waiting for unit multi-user.target container-test-run-monitoring> machine2: (finished: waiting for unit multi-user.target, in 0.01 seconds) container-test-run-monitoring> machine1: waiting for unit alloy container-test-run-monitoring> machine1: (finished: waiting for unit alloy, in 0.01 seconds) container-test-run-monitoring> machine2: waiting for unit alloy container-test-run-monitoring> machine2: (finished: waiting for unit alloy, in 0.01 seconds) container-test-run-monitoring> machine1: must succeed: test -f /etc/alloy/config.alloy container-test-run-monitoring> machine1: (finished: must succeed: test -f /etc/alloy/config.alloy, in 0.00 seconds) container-test-run-monitoring> machine2: must succeed: test -f /etc/alloy/config.alloy container-test-run-monitoring> machine2: (finished: must succeed: test -f /etc/alloy/config.alloy, in 0.00 seconds) container-test-run-monitoring> machine2: must succeed: test -f /etc/alloy/local-extension.alloy container-test-run-monitoring> machine2: (finished: must succeed: test -f /etc/alloy/local-extension.alloy, in 0.01 seconds) container-test-run-monitoring> machine1: must succeed: systemctl show alloy --property=ExecStart | grep -F '/etc/alloy' container-test-run-monitoring> machine1: (finished: must succeed: systemctl show alloy --property=ExecStart | grep -F '/etc/alloy', in 0.01 seconds) container-test-run-monitoring> machine2: must succeed: systemctl show alloy --property=ExecStart | grep -F '/etc/alloy' container-test-run-monitoring> machine2: (finished: must succeed: systemctl show alloy --property=ExecStart | grep -F '/etc/alloy', in 0.01 seconds) container-test-run-monitoring> machine1: waiting for unit loki container-test-run-monitoring> machine1: (finished: waiting for unit loki, in 0.01 seconds) container-test-run-monitoring> machine1: waiting for unit mimir container-test-run-monitoring> machine1: (finished: waiting for unit mimir, in 0.01 seconds) container-test-run-monitoring> machine1: waiting for unit grafana container-test-run-monitoring> machine1: (finished: waiting for unit grafana, in 0.01 seconds) container-test-run-monitoring> machine2: must succeed: test "$(grep -c '^loki.source.journal ' /etc/alloy/config.alloy)" = 1 container-test-run-monitoring> machine2: (finished: must succeed: test "$(grep -c '^loki.source.journal ' /etc/alloy/config.alloy)" = 1, in 0.01 seconds) container-test-run-monitoring> machine2: must succeed: cat /etc/alloy/config.alloy container-test-run-monitoring> machine2: (finished: must succeed: cat /etc/alloy/config.alloy, in 0.01 seconds) container-test-run-monitoring> (finished: run the VM test script, in 10.76 seconds) container-test-run-monitoring> test script finished in 10.92s container-test-run-monitoring> cleanup container-test-run-monitoring> kill NspawnMachine (pid 52) container-test-run-monitoring> kill NspawnMachine (pid 53) container-test-run-monitoring> Container machine1 terminated by signal KILL. container-test-run-monitoring> (finished: cleanup, in 0.18 seconds) container-test-run-monitoring> Container machine2 terminated by signal KILL. upload niks3: ok