these 58 derivations will be built: /nix/store/43sl1f5z97kl85s3r4hifqczx382psjw-users-groups.json.drv /nix/store/ndlkbkhkd1id651zd1g3lbhsdkjbsckw-decrypt-age-secrets.drv /nix/store/0j9ir0sx4smdi01kxv1gdbs7c0i3flkp-dry-activate.drv /nix/store/ccigflg3f1nlikhmh5az14v5kc7hq72k-etc-hostname.drv /nix/store/k1cviy5n7iyqb260fpsfcpm2kjksmv2f-etc-alloy-local-extension.alloy.drv /nix/store/a21hdxcg48y17fg82vld6aqpajfvflvl-string-hosts.drv /nix/store/w5dfbqvfaaicsmh4f48c22jsd0b0dj1k-extra-hosts.drv /nix/store/l07sq5jscsmkqmm6g34ykdk14vbxbww2-hosts.drv /nix/store/p95bl2danxglpm6xw4506di4n0p5zar1-vars-check-monitoring.drv /nix/store/k4rq6lw4w56w0ccwhy2iia70cn0rxxhn-X-Reload-Triggers-alloy.drv /nix/store/ng7daslv1m3f5dgvgxjpjpn8h12v8xgc-unit-alloy.service.drv /nix/store/zg8pf34mw4nm6w62dijnj61jnmln4nnc-system-units.drv /nix/store/0vv73pw5pxbgnm8dd9y28g9czx2rza80-etc.drv /nix/store/bpzrbhnp5bgynhxy2yacmq1v77s490bs-config.yaml.drv /nix/store/df47rgwcbcad36z163h8q4xrh7qn3w0y-unit-mimir.service.drv /nix/store/6qs8h89w6r95zgdyhck7glzlzbdv5aym-postgres-db-restore-command-grafana.drv /nix/store/ckpxzcq6zlv9wbww3ajg10j6902r07dq-state-commands.drv /nix/store/md7xsjkz0pyvfg2md3gvdrsdabg2p8p5-system-path.drv /nix/store/fll08w2pz2faniabwj9krq8qsfjrx8n4-dbus-1.drv /nix/store/yncqpmqiisvc410a6sl8wanj1pxwjwmj-X-Restart-Triggers-dbus-broker.drv /nix/store/j1v667vx0w9pz9kr7mdgaf3vr41a4lby-unit-dbus-broker.service.drv /nix/store/3nvkwlb699h6wzi7ab5jkzaxjfsmwssd-unit-script-postgresql-post-start.drv /nix/store/kvc98cfs9lq2lhpqisr1dkqygndj87yw-unit-postgresql.service.drv /nix/store/i26rr93ys7hbqika2y7acv62w3kclg88-nginx.conf.drv /nix/store/9j375ff96xxsxb96k0bfv1qdkk2x0gxj-unit-script-nginx-pre-start.drv /nix/store/lcm67q4h736qf1574k5xfzy3pypkmavz-unit-nginx.service.drv /nix/store/fkmbspcm2drkcwyqffqw1n66n9jfsyzs-firewall-start.drv /nix/store/xvw9rnkx8h81axmcpbkrpqgmhzxfqkix-firewall-reload.drv /nix/store/m6li7652dawkr7wqcfhg29j4lyjdf2g9-unit-firewall.service.drv /nix/store/gzcmh5z9pjrjbav96zsaa85nvi6afygi-loki-config.json.drv /nix/store/2m3gfs1nqykzsr97ldqw1qkiy951pkwk-validate-loki-conf.drv /nix/store/mjz5f8cisp66vg6m2barvr89bpd7nvqx-unit-loki.service.drv /nix/store/5hw7r04p9r5jdh076cmzn7cyw0mv695r-dashboard.yaml.drv /nix/store/lkd37gr78ych6rzrvgrp85naqdnp8mh6-datasource.yaml.drv /nix/store/vfqwfjfmv1zv5pg9mxb623868mm7ki5n-grafana-provisioning.drv /nix/store/gfxvxhdwasgdb0r9dj0s1s9i5knkrp33-config.ini.drv /nix/store/my49va3bdm1z0bacw53vmffmlk98hw5d-unit-grafana.service.drv /nix/store/khw8v9lj9w1yvlvxvci4sb4pi1h3ccl7-X-Reload-Triggers-alloy.drv /nix/store/pp339rparbdsvnigrk4i7jgljq7rvhj6-unit-alloy.service.drv /nix/store/dqsqgmp1dz9p667811xx8ljlybmlnk5z-system-units.drv /nix/store/q3ikr8q9q5b26ihq9wy9pm60lc5vz5z4-etc-hostname.drv /nix/store/a6qvaca707sdar02jw60p3q7n78l62s8-unit-dbus-broker.service.drv /nix/store/sycr2nv3p1cyc5xsibmkv473xix05b4i-user-units.drv /nix/store/r6szj0k7gfbmi5f9frlpbv1i217sz67p-string-hosts.drv /nix/store/wsyypbygmlbihvr2x5kkiaspvmmi67in-hosts.drv /nix/store/2fq6ac60hf418vzrn55scjd491i7swn2-etc.drv /nix/store/j1jnr6njvshm5vx9456fmlb7yyiqz6pb-decrypt-age-secrets.drv /nix/store/b5c5ilv4v7ws48a4gm3bxh3w5wqd2m60-dry-activate.drv /nix/store/p0r6jssz113ciry7x9jj5na0x5qr0a79-activate.drv /nix/store/jbf67is0yws04dsihvkdcjpy9df6jryd-nixos-system-machine2-test.drv /nix/store/7bkfgfv28j7ffaiw94zdvfl5mvk5piws-run-machine2-nspawn.drv /nix/store/cr2dzdf3zpvag4zxhm1w1xi49yhmkidm-test-script.drv /nix/store/9q9jy82myzlfmynjb2nr7dcsk3ihdx5k-activate.drv /nix/store/6j6lkq9c1vrwzr23mbslgc7qds5bycwa-nixos-system-machine1-test.drv /nix/store/pz2a5csifkiwf8b3qc2y9hcrx96zcn7l-run-machine1-nspawn.drv /nix/store/3477prj37kf1xpnn9c6gc9hd797smr35-driverConfiguration.json.drv /nix/store/m536214psykm7k998yj14mg5dyznv8wi-nixos-test-driver-monitoring.drv /nix/store/pswdisnq32kr6zv9p60scp6rbriib158-container-test-run-monitoring.drv these 9 paths will be fetched (627.2 MiB download, 2.3 GiB unpacked): /nix/store/a7ngc3mhdf5yhn9qwvpdh4nq3wk77q3i-contactPoints.yaml /nix/store/imrqc8lv29sg4q86kvwnfiyyvvli625v-grafana-13.1.3 /nix/store/qpi5rkz7cv9qswd9l1ylqypdipjzjgs2-grafana-alloy-1.17.1 /nix/store/bjr32xsn2q8vp3wcwc8h5hrpbqn5r2sg-grafana-loki-3.7.6 /nix/store/gww8by1dggybygcqgp12ssy3vwng2wsp-mimir-3.1.4 /nix/store/x7zqlzkms8qb1d4srx15n8v1d4nc7ska-muteTimings.yaml /nix/store/sv1snya8l8dvw8shy38s71pygjzjqpyr-policies.yaml /nix/store/whfnf03wz0h188wa4z45a6d3aw6667k2-rules.yaml /nix/store/kmkvz29kzsf7c376wzd864a5w2083nn2-templates.yaml building '/nix/store/cr2dzdf3zpvag4zxhm1w1xi49yhmkidm-test-script.drv' building '/nix/store/k1cviy5n7iyqb260fpsfcpm2kjksmv2f-etc-alloy-local-extension.alloy.drv' building '/nix/store/ccigflg3f1nlikhmh5az14v5kc7hq72k-etc-hostname.drv' building '/nix/store/q3ikr8q9q5b26ihq9wy9pm60lc5vz5z4-etc-hostname.drv' building '/nix/store/i26rr93ys7hbqika2y7acv62w3kclg88-nginx.conf.drv' building '/nix/store/6qs8h89w6r95zgdyhck7glzlzbdv5aym-postgres-db-restore-command-grafana.drv' building '/nix/store/w5dfbqvfaaicsmh4f48c22jsd0b0dj1k-extra-hosts.drv' building '/nix/store/a21hdxcg48y17fg82vld6aqpajfvflvl-string-hosts.drv' building '/nix/store/r6szj0k7gfbmi5f9frlpbv1i217sz67p-string-hosts.drv' building '/nix/store/khw8v9lj9w1yvlvxvci4sb4pi1h3ccl7-X-Reload-Triggers-alloy.drv' nginx.conf> structuredAttrs is enabled building '/nix/store/fkmbspcm2drkcwyqffqw1n66n9jfsyzs-firewall-start.drv' building '/nix/store/3nvkwlb699h6wzi7ab5jkzaxjfsmwssd-unit-script-postgresql-post-start.drv' building '/nix/store/k4rq6lw4w56w0ccwhy2iia70cn0rxxhn-X-Reload-Triggers-alloy.drv' building '/nix/store/l07sq5jscsmkqmm6g34ykdk14vbxbww2-hosts.drv' building '/nix/store/xvw9rnkx8h81axmcpbkrpqgmhzxfqkix-firewall-reload.drv' building '/nix/store/wsyypbygmlbihvr2x5kkiaspvmmi67in-hosts.drv' building '/nix/store/kvc98cfs9lq2lhpqisr1dkqygndj87yw-unit-postgresql.service.drv' building '/nix/store/bpzrbhnp5bgynhxy2yacmq1v77s490bs-config.yaml.drv' building '/nix/store/ckpxzcq6zlv9wbww3ajg10j6902r07dq-state-commands.drv' building '/nix/store/p95bl2danxglpm6xw4506di4n0p5zar1-vars-check-monitoring.drv' building '/nix/store/43sl1f5z97kl85s3r4hifqczx382psjw-users-groups.json.drv' config.yaml> structuredAttrs is enabled nginx.conf> nginx.conf> ==================== Results =================== nginx.conf> No issues found. nginx.conf> nginx.conf> ==================== Summary =================== nginx.conf> Total issues: nginx.conf> Unspecified: 0 nginx.conf> Low: 0 nginx.conf> Medium: 0 nginx.conf> High: 0 nginx.conf> unit-postgresql.service> structuredAttrs is enabled building '/nix/store/m6li7652dawkr7wqcfhg29j4lyjdf2g9-unit-firewall.service.drv' vars-check-monitoring> Running vars check using Nix-based executor... vars-check-monitoring> ✓ All vars checks completed successfully building '/nix/store/9j375ff96xxsxb96k0bfv1qdkk2x0gxj-unit-script-nginx-pre-start.drv' unit-firewall.service> structuredAttrs is enabled building '/nix/store/5hw7r04p9r5jdh076cmzn7cyw0mv695r-dashboard.yaml.drv' building '/nix/store/lkd37gr78ych6rzrvgrp85naqdnp8mh6-datasource.yaml.drv' building '/nix/store/j1jnr6njvshm5vx9456fmlb7yyiqz6pb-decrypt-age-secrets.drv' building '/nix/store/ndlkbkhkd1id651zd1g3lbhsdkjbsckw-decrypt-age-secrets.drv' building '/nix/store/gzcmh5z9pjrjbav96zsaa85nvi6afygi-loki-config.json.drv' config.yaml> dashboard.yaml> structuredAttrs is enabled datasource.yaml> structuredAttrs is enabled building '/nix/store/lcm67q4h736qf1574k5xfzy3pypkmavz-unit-nginx.service.drv' building '/nix/store/0j9ir0sx4smdi01kxv1gdbs7c0i3flkp-dry-activate.drv' building '/nix/store/b5c5ilv4v7ws48a4gm3bxh3w5wqd2m60-dry-activate.drv' dashboard.yaml> unit-nginx.service> structuredAttrs is enabled datasource.yaml> building '/nix/store/vfqwfjfmv1zv5pg9mxb623868mm7ki5n-grafana-provisioning.drv' building '/nix/store/2m3gfs1nqykzsr97ldqw1qkiy951pkwk-validate-loki-conf.drv' validate-loki-conf> level=info ts=2026-08-17T10:02:14.825206613Z caller=main.go:109 msg="config is valid" building '/nix/store/mjz5f8cisp66vg6m2barvr89bpd7nvqx-unit-loki.service.drv' building '/nix/store/ng7daslv1m3f5dgvgxjpjpn8h12v8xgc-unit-alloy.service.drv' building '/nix/store/pp339rparbdsvnigrk4i7jgljq7rvhj6-unit-alloy.service.drv' unit-alloy.service> structuredAttrs is enabled unit-loki.service> structuredAttrs is enabled unit-alloy.service> structuredAttrs is enabled building '/nix/store/zg8pf34mw4nm6w62dijnj61jnmln4nnc-system-units.drv' building '/nix/store/0vv73pw5pxbgnm8dd9y28g9czx2rza80-etc.drv' building '/nix/store/df47rgwcbcad36z163h8q4xrh7qn3w0y-unit-mimir.service.drv' unit-mimir.service> structuredAttrs is enabled building '/nix/store/p0r6jssz113ciry7x9jj5na0x5qr0a79-activate.drv' building '/nix/store/jbf67is0yws04dsihvkdcjpy9df6jryd-nixos-system-machine2-test.drv' nixos-system-machine2-test> structuredAttrs is enabled building '/nix/store/7bkfgfv28j7ffaiw94zdvfl5mvk5piws-run-machine2-nspawn.drv' building '/nix/store/gfxvxhdwasgdb0r9dj0s1s9i5knkrp33-config.ini.drv' building '/nix/store/md7xsjkz0pyvfg2md3gvdrsdabg2p8p5-system-path.drv' system-path> structuredAttrs is enabled system-path> created 1873 symlinks in user environment building '/nix/store/my49va3bdm1z0bacw53vmffmlk98hw5d-unit-grafana.service.drv' unit-grafana.service> structuredAttrs is enabled building '/nix/store/fll08w2pz2faniabwj9krq8qsfjrx8n4-dbus-1.drv' building '/nix/store/yncqpmqiisvc410a6sl8wanj1pxwjwmj-X-Restart-Triggers-dbus-broker.drv' building '/nix/store/a6qvaca707sdar02jw60p3q7n78l62s8-unit-dbus-broker.service.drv' building '/nix/store/j1v667vx0w9pz9kr7mdgaf3vr41a4lby-unit-dbus-broker.service.drv' unit-dbus-broker.service> structuredAttrs is enabled building '/nix/store/sycr2nv3p1cyc5xsibmkv473xix05b4i-user-units.drv' unit-dbus-broker.service> structuredAttrs is enabled building '/nix/store/dqsqgmp1dz9p667811xx8ljlybmlnk5z-system-units.drv' building '/nix/store/2fq6ac60hf418vzrn55scjd491i7swn2-etc.drv' building '/nix/store/9q9jy82myzlfmynjb2nr7dcsk3ihdx5k-activate.drv' building '/nix/store/6j6lkq9c1vrwzr23mbslgc7qds5bycwa-nixos-system-machine1-test.drv' nixos-system-machine1-test> structuredAttrs is enabled building '/nix/store/pz2a5csifkiwf8b3qc2y9hcrx96zcn7l-run-machine1-nspawn.drv' building '/nix/store/3477prj37kf1xpnn9c6gc9hd797smr35-driverConfiguration.json.drv' driverConfiguration.json> structuredAttrs is enabled building '/nix/store/m536214psykm7k998yj14mg5dyznv8wi-nixos-test-driver-monitoring.drv' nixos-test-driver-monitoring> Running type check (enable/disable: config.skipTypeCheck) nixos-test-driver-monitoring> See https://nixos.org/manual/nixos/stable/#test-opt-skipTypeCheck nixos-test-driver-monitoring> All checks passed! nixos-test-driver-monitoring> Linting test script (enable/disable: config.skipLint) nixos-test-driver-monitoring> See https://nixos.org/manual/nixos/stable/#test-opt-skipLint nixos-test-driver-monitoring> All checks passed! building '/nix/store/pswdisnq32kr6zv9p60scp6rbriib158-container-test-run-monitoring.drv' container-test-run-monitoring> Machine state will be reset. To keep it, pass --keep-machine-state container-test-run-monitoring> start all VLans container-test-run-monitoring> (finished: start all VLans, in 0.00 seconds) container-test-run-monitoring> container-test-run-monitoring> Test will time out and terminate in 3600.0 seconds container-test-run-monitoring> run the VM test script container-test-run-monitoring> additionally exposed symbols: container-test-run-monitoring> machine1, machine2, container-test-run-monitoring> vlan1, container-test-run-monitoring> start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh container-test-run-monitoring> start all VMs container-test-run-monitoring> machine2: systemd-nspawn running (pid 53) container-test-run-monitoring> machine1: systemd-nspawn running (pid 52) container-test-run-monitoring> machine2: Waiting for journal at /build/vm-state-machine2/var/log/journal... container-test-run-monitoring> machine1: Waiting for journal at /build/vm-state-machine1/var/log/journal... container-test-run-monitoring> (finished: start all VMs, in 0.00 seconds) container-test-run-monitoring> machine1: waiting for unit multi-user.target container-test-run-monitoring> nixos-nspawn(machine2): TAP vde-tap1 not found; container will be isolated from VDE container-test-run-monitoring> nixos-nspawn(machine2): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths. container-test-run-monitoring> nixos-nspawn(machine1): TAP vde-tap1 not found; container will be isolated from VDE container-test-run-monitoring> nixos-nspawn(machine1): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths. container-test-run-monitoring> Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file. container-test-run-monitoring> Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file. container-test-run-monitoring> ░ Spawning container machine2 on /build/vm-state-machine2. container-test-run-monitoring> ░ Spawning container machine1 on /build/vm-state-machine1. container-test-run-monitoring> machine1 # [1123633.552545] machine1 systemd-journald[124]: Journal started container-test-run-monitoring> machine2 # [1123633.552185] machine2 systemd-journald[105]: Journal started container-test-run-monitoring> machine1 # [1123633.552585] machine1 systemd-journald[124]: Runtime Journal (/run/log/journal/58b123ee3e9c4e38b38b35b7f6c233de) is 8M, max 4G, 3.9G free. container-test-run-monitoring> machine2 # [1123633.552269] machine2 systemd-journald[105]: Runtime Journal (/run/log/journal/5f037e2ee03f4ffd8df1afc3b6500498) is 8M, max 4G, 3.9G free. container-test-run-monitoring> machine1 # [1123633.555943] machine1 systemd[1]: Finished Create Static Device Nodes in /dev gracefully. container-test-run-monitoring> machine2 # [1123633.556297] machine2 systemd[1]: Finished Create Static Device Nodes in /dev gracefully. container-test-run-monitoring> machine1 # [1123633.562588] machine1 systemd[1]: Starting Flush Journal to Persistent Storage... container-test-run-monitoring> machine2 # [1123633.562942] machine2 systemd[1]: Starting Flush Journal to Persistent Storage... container-test-run-monitoring> machine1 # [1123633.563107] machine1 systemd[1]: Starting Network Name Resolution... container-test-run-monitoring> machine2 # [1123633.563421] machine2 systemd[1]: Starting Network Name Resolution... container-test-run-monitoring> machine1 # [1123633.563512] machine1 systemd[1]: Starting Create Static Device Nodes in /dev... container-test-run-monitoring> machine2 # [1123633.563804] machine2 systemd[1]: Starting Create Static Device Nodes in /dev... container-test-run-monitoring> machine1 # [1123633.568956] machine1 systemd-journald[124]: Time spent on flushing to /var/log/journal/58b123ee3e9c4e38b38b35b7f6c233de is 1.397ms for 6 entries. container-test-run-monitoring> machine2 # [1123633.568953] machine2 systemd-journald[105]: Time spent on flushing to /var/log/journal/5f037e2ee03f4ffd8df1afc3b6500498 is 1.303ms for 6 entries. container-test-run-monitoring> machine1 # [1123633.568956] machine1 systemd-journald[124]: System Journal (/var/log/journal/58b123ee3e9c4e38b38b35b7f6c233de) is 512B, max 4G, 3.9G free. container-test-run-monitoring> machine2 # [1123633.568953] machine2 systemd-journald[105]: System Journal (/var/log/journal/5f037e2ee03f4ffd8df1afc3b6500498) is 512B, max 4G, 3.9G free. container-test-run-monitoring> machine1 # [1123633.582347] machine1 systemd[1]: Finished Create Static Device Nodes in /dev. container-test-run-monitoring> machine2 # [1123633.581964] machine2 systemd[1]: Finished Create Static Device Nodes in /dev. container-test-run-monitoring> machine1 # [1123633.582860] machine1 systemd[1]: Reached target Preparation for Local File Systems. container-test-run-monitoring> machine2 # [1123633.582702] machine2 systemd[1]: Reached target Preparation for Local File Systems. container-test-run-monitoring> machine1 # [1123633.582922] machine1 systemd[1]: Reached target Local File Systems. container-test-run-monitoring> machine2 # [1123633.582763] machine2 systemd[1]: Reached target Local File Systems. container-test-run-monitoring> machine1 # [1123633.583490] machine1 systemd[1]: Listening on Boot Loader Control Service Socket. container-test-run-monitoring> machine1 # [1123633.583522] machine1 systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container container-test-run-monitoring> machine2 # [1123633.583491] machine2 systemd[1]: Listening on Boot Loader Control Service Socket. container-test-run-monitoring> machine2 # [1123633.583522] machine2 systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container container-test-run-monitoring> machine1 # [1123633.584083] machine1 systemd[1]: Starting Save Transient machine-id to Disk... container-test-run-monitoring> machine1 # [1123633.584102] machine1 systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys container-test-run-monitoring> machine2 # [1123633.584196] machine2 systemd[1]: Starting Save Transient machine-id to Disk... container-test-run-monitoring> machine1 # [1123633.603900] machine1 systemd[1]: Finished Flush Journal to Persistent Storage. container-test-run-monitoring> machine2 # [1123633.584215] machine2 systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys container-test-run-monitoring> machine1 # [1123633.604642] machine1 systemd[1]: Starting Create System Files and Directories... container-test-run-monitoring> machine2 # [1123633.602502] machine2 systemd[1]: Finished Flush Journal to Persistent Storage. container-test-run-monitoring> machine1 # [1123633.648415] machine1 systemd-tmpfiles[177]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted container-test-run-monitoring> machine2 # [1123633.603729] machine2 systemd[1]: Starting Create System Files and Directories... container-test-run-monitoring> machine1 # [1123633.648664] machine1 systemd-tmpfiles[177]: fchmod() of /var/log/journal failed: Operation not permitted container-test-run-monitoring> machine2 # [1123633.648420] machine2 systemd-tmpfiles[159]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted container-test-run-monitoring> machine1 # [1123633.648813] machine1 systemd-tmpfiles[177]: fchmod() of /var/log/journal/58b123ee3e9c4e38b38b35b7f6c233de failed: Operation not permitted container-test-run-monitoring> machine2 # [1123633.648676] machine2 systemd-tmpfiles[159]: fchmod() of /var/log/journal failed: Operation not permitted container-test-run-monitoring> machine1 # [1123633.649031] machine1 systemd-tmpfiles[177]: fchmod() of /run/log/journal failed: Operation not permitted container-test-run-monitoring> machine2 # [1123633.648835] machine2 systemd-tmpfiles[159]: fchmod() of /var/log/journal/5f037e2ee03f4ffd8df1afc3b6500498 failed: Operation not permitted container-test-run-monitoring> machine1 # [1123633.650489] machine1 systemd[1]: Finished Create System Files and Directories. container-test-run-monitoring> machine2 # [1123633.649062] machine2 systemd-tmpfiles[159]: fchmod() of /run/log/journal failed: Operation not permitted container-test-run-monitoring> machine1 # [1123633.651408] machine1 systemd[1]: Starting Rebuild Journal Catalog... container-test-run-monitoring> machine2 # [1123633.650512] machine2 systemd[1]: Finished Create System Files and Directories. container-test-run-monitoring> machine1 # [1123633.652023] machine1 systemd[1]: Starting Record System Boot/Shutdown in UTMP... container-test-run-monitoring> machine2 # [1123633.651436] machine2 systemd[1]: Starting Rebuild Journal Catalog... container-test-run-monitoring> machine1 # [1123633.675708] machine1 systemd[1]: Finished Firewall. container-test-run-monitoring> machine2 # [1123633.652020] machine2 systemd[1]: Starting Record System Boot/Shutdown in UTMP... container-test-run-monitoring> machine1 # [1123633.675834] machine1 systemd[1]: Reached target Preparation for Network. container-test-run-monitoring> machine2 # [1123633.673465] machine2 systemd[1]: Finished Firewall. container-test-run-monitoring> machine1 # [1123633.676034] machine1 systemd[1]: Listening on Network Management Resolve Hook Socket. container-test-run-monitoring> machine2 # [1123633.673542] machine2 systemd[1]: Reached target Preparation for Network. container-test-run-monitoring> machine1 # [1123633.676788] machine1 systemd[1]: Starting Network Management... container-test-run-monitoring> machine2 # [1123633.673775] machine2 systemd[1]: Listening on Network Management Resolve Hook Socket. container-test-run-monitoring> machine1 # [1123633.679697] machine1 systemd[1]: Finished Record System Boot/Shutdown in UTMP. container-test-run-monitoring> machine2 # [1123633.674658] machine2 systemd[1]: Starting Network Management... container-test-run-monitoring> machine1 # [1123633.690409] machine1 systemd[1]: Finished Rebuild Journal Catalog. container-test-run-monitoring> machine2 # [1123633.679702] machine2 systemd[1]: Finished Record System Boot/Shutdown in UTMP. container-test-run-monitoring> machine1 # [1123633.691157] machine1 systemd[1]: Starting Update is Completed... container-test-run-monitoring> machine2 # [1123633.690627] machine2 systemd[1]: Finished Rebuild Journal Catalog. container-test-run-monitoring> machine1 # [1123633.698523] machine1 systemd[1]: Finished Update is Completed. container-test-run-monitoring> machine2 # [1123633.691304] machine2 systemd[1]: Starting Update is Completed... container-test-run-monitoring> machine2 # [1123633.698671] machine2 systemd[1]: Finished Update is Completed. container-test-run-monitoring> machine1 # [1123634.066686] machine1 systemd-networkd[235]: Failed to increase receive buffer size for general netlink socket, ignoring: Operation not permitted container-test-run-monitoring> machine1 # [1123634.066816] machine1 systemd-networkd[235]: Failed to increase receive buffer size for nftables netlink socket, ignoring: Operation not permitted container-test-run-monitoring> machine1 # [1123634.079242] machine1 systemd-networkd[235]: /etc/systemd/network/99-ethernet-default-dhcp.network: No valid settings found in the [Match] section, ignoring file. To match all interfaces, add Name=* in the [Match] section. container-test-run-monitoring> machine1 # [1123634.079475] machine1 systemd-networkd[235]: /etc/systemd/network/99-wireless-client-dhcp.network: No valid settings found in the [Match] section, ignoring file. To match all interfaces, add Name=* in the [Match] section. container-test-run-monitoring> machine1 # [1123634.079729] machine1 systemd-networkd[235]: lo: Link UP container-test-run-monitoring> machine1 # [1123634.079733] machine1 systemd-networkd[235]: lo: Gained carrier container-test-run-monitoring> machine1 # [1123634.080060] machine1 systemd-networkd[235]: eth1: Configuring with /etc/systemd/network/40-eth1.network. container-test-run-monitoring> machine1 # [1123634.080655] machine1 systemd[1]: Started Network Management. container-test-run-monitoring> machine1 # [1123634.080739] machine1 systemd-networkd[235]: eth1: Link UP container-test-run-monitoring> machine1 # [1123634.080953] machine1 systemd-networkd[235]: eth1: Gained carrier container-test-run-monitoring> machine1 # [1123634.081857] machine1 systemd[1]: Starting Enable Persistent Storage in systemd-networkd... container-test-run-monitoring> machine1 # [1123634.108430] machine1 systemd[1]: Finished Enable Persistent Storage in systemd-networkd. container-test-run-monitoring> machine1 # [1123634.173220] machine1 systemd[1]: Finished Save Transient machine-id to Disk. container-test-run-monitoring> machine1 # [1123634.314274] machine1 systemd-resolved[142]: Positive Trust Anchors: container-test-run-monitoring> machine1 # [1123634.314285] machine1 systemd-resolved[142]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d container-test-run-monitoring> machine1 # [1123634.314289] machine1 systemd-resolved[142]: . IN DS 38696 8 2 683d2d0acb8c9b712a1948b27f741219298d0a450d612c483af444a4c0fb2b16 container-test-run-monitoring> machine1 # [1123634.314309] machine1 systemd-resolved[142]: Negative trust anchors: home.arpa 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 170.0.0.192.in-addr.arpa 171.0.0.192.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa ipv4only.arpa resolver.arpa corp home internal intranet lan local private test container-test-run-monitoring> machine1 # [1123634.330157] machine1 systemd-resolved[142]: Using system hostname 'machine1'. container-test-run-monitoring> machine1 # [1123634.331279] machine1 systemd[1]: Started Network Name Resolution. container-test-run-monitoring> machine1 # [1123634.331339] machine1 systemd[1]: Reached target Network. container-test-run-monitoring> machine1 # [1123634.331372] machine1 systemd[1]: Reached target Network is Online. container-test-run-monitoring> machine1 # [1123634.331402] machine1 systemd[1]: Reached target System Initialization. container-test-run-monitoring> machine1 # [1123634.331438] machine1 systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container container-test-run-monitoring> machine1 # [1123634.331460] machine1 systemd[1]: Started Daily Cleanup of Temporary Directories. container-test-run-monitoring> machine1 # [1123634.331473] machine1 systemd[1]: Reached target Timer Units. container-test-run-monitoring> machine1 # [1123634.331565] machine1 systemd[1]: Listening on D-Bus System Message Bus Socket. container-test-run-monitoring> machine1 # [1123634.331666] machine1 systemd[1]: Listening on Nix Daemon Socket. container-test-run-monitoring> machine1 # [1123634.331749] machine1 systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. container-test-run-monitoring> machine1 # [1123634.331760] machine1 systemd[1]: Reached target Socket Units. container-test-run-monitoring> machine1 # [1123634.331786] machine1 systemd[1]: Reached target Basic System. container-test-run-monitoring> machine2 # [1123634.091702] machine2 systemd-networkd[214]: Failed to increase receive buffer size for general netlink socket, ignoring: Operation not permitted container-test-run-monitoring> machine2 # [1123634.091790] machine2 systemd-networkd[214]: Failed to increase receive buffer size for nftables netlink socket, ignoring: Operation not permitted container-test-run-monitoring> machine2 # [1123634.099438] machine2 systemd-networkd[214]: /etc/systemd/network/99-ethernet-default-dhcp.network: No valid settings found in the [Match] section, ignoring file. To match all interfaces, add Name=* in the [Match] section. container-test-run-monitoring> machine2 # [1123634.099649] machine2 systemd-networkd[214]: /etc/systemd/network/99-wireless-client-dhcp.network: No valid settings found in the [Match] section, ignoring file. To match all interfaces, add Name=* in the [Match] section. container-test-run-monitoring> machine2 # [1123634.099753] machine2 systemd-networkd[214]: lo: Link UP container-test-run-monitoring> machine2 # [1123634.099757] machine2 systemd-networkd[214]: lo: Gained carrier container-test-run-monitoring> machine2 # [1123634.099920] machine2 systemd-networkd[214]: eth1: Configuring with /etc/systemd/network/40-eth1.network. container-test-run-monitoring> machine2 # [1123634.100222] machine2 systemd[1]: Started Network Management. container-test-run-monitoring> machine2 # [1123634.102145] machine2 systemd-networkd[214]: eth1: Link UP container-test-run-monitoring> machine2 # [1123634.102315] machine2 systemd-networkd[214]: eth1: Gained carrier container-test-run-monitoring> machine2 # [1123634.102626] machine2 systemd[1]: Starting Enable Persistent Storage in systemd-networkd... container-test-run-monitoring> machine2 # [1123634.119277] machine2 systemd[1]: Finished Enable Persistent Storage in systemd-networkd. container-test-run-monitoring> machine2 # [1123634.172980] machine2 systemd[1]: Finished Save Transient machine-id to Disk. container-test-run-monitoring> machine2 # [1123634.296687] machine2 systemd-resolved[123]: Positive Trust Anchors: container-test-run-monitoring> machine2 # [1123634.296713] machine2 systemd-resolved[123]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d container-test-run-monitoring> machine2 # [1123634.296718] machine2 systemd-resolved[123]: . IN DS 38696 8 2 683d2d0acb8c9b712a1948b27f741219298d0a450d612c483af444a4c0fb2b16 container-test-run-monitoring> machine2 # [1123634.296739] machine2 systemd-resolved[123]: Negative trust anchors: home.arpa 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 170.0.0.192.in-addr.arpa 171.0.0.192.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa ipv4only.arpa resolver.arpa corp home internal intranet lan local private test container-test-run-monitoring> machine2 # [1123634.311347] machine2 systemd-resolved[123]: Using system hostname 'machine2'. container-test-run-monitoring> machine2 # [1123634.312730] machine2 systemd[1]: Started Network Name Resolution. container-test-run-monitoring> machine2 # [1123634.312807] machine2 systemd[1]: Reached target Network. container-test-run-monitoring> machine2 # [1123634.312863] machine2 systemd[1]: Reached target System Initialization. container-test-run-monitoring> machine2 # [1123634.312911] machine2 systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container container-test-run-monitoring> machine2 # [1123634.312983] machine2 systemd[1]: Started Daily Cleanup of Temporary Directories. container-test-run-monitoring> machine2 # [1123634.313018] machine2 systemd[1]: Reached target Timer Units. container-test-run-monitoring> machine2 # [1123634.313137] machine2 systemd[1]: Listening on D-Bus System Message Bus Socket. container-test-run-monitoring> machine2 # [1123634.313291] machine2 systemd[1]: Listening on Nix Daemon Socket. container-test-run-monitoring> machine2 # [1123634.313405] machine2 systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. container-test-run-monitoring> machine2 # [1123634.313422] machine2 systemd[1]: Reached target Socket Units. container-test-run-monitoring> machine2 # [1123634.313455] machine2 systemd[1]: Reached target Basic System. container-test-run-monitoring> machine2 # [1123634.314631] machine2 systemd[1]: Started alloy.service. container-test-run-monitoring> machine2 # [1123634.315285] machine2 systemd[1]: Starting Import lastlog data into lastlog2 database... container-test-run-monitoring> machine2 # [1123634.316096] machine2 systemd[1]: Starting Name Service Cache Daemon (nsncd)... container-test-run-monitoring> machine2 # [1123634.317211] machine2 systemd[1]: Starting D-Bus System Message Bus... container-test-run-monitoring> machine1 # [1123634.342406] machine1 systemd[1]: Started alloy.service. container-test-run-monitoring> machine1 # [1123634.343216] machine1 systemd[1]: Starting Import lastlog data into lastlog2 database... container-test-run-monitoring> machine1 # [1123634.343876] machine1 systemd[1]: Started Loki Service Daemon. container-test-run-monitoring> machine1 # [1123634.345495] machine1 systemd[1]: Started mimir Service Daemon. container-test-run-monitoring> machine1 # [1123634.348123] machine1 systemd[1]: Starting Nginx Web Server... container-test-run-monitoring> machine1 # [1123634.349194] machine1 systemd[1]: Starting Name Service Cache Daemon (nsncd)... container-test-run-monitoring> machine1 # [1123634.350305] machine1 systemd[1]: Starting PostgreSQL Server... container-test-run-monitoring> machine1 # [1123634.351765] machine1 systemd[1]: Starting D-Bus System Message Bus... container-test-run-monitoring> machine1 # [1123634.361094] machine1 systemd[1]: Finished Import lastlog data into lastlog2 database. container-test-run-monitoring> machine1 # [1123634.516327] machine1 nsncd[251]: Aug 17 10:02:24.739 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" container-test-run-monitoring> machine1 # [1123634.516427] machine1 systemd[1]: Started Name Service Cache Daemon (nsncd). container-test-run-monitoring> machine1 # [1123634.516483] machine1 systemd[1]: Reached target Host and Network Name Lookups. container-test-run-monitoring> machine1 # [1123634.516519] machine1 systemd[1]: Reached target User and Group Name Lookups. container-test-run-monitoring> machine1 # [1123634.517366] machine1 systemd[1]: Starting User Login Management... container-test-run-monitoring> machine1 # [1123634.517925] machine1 systemd[1]: Starting Permit User Sessions... container-test-run-monitoring> machine1 # [1123634.544931] machine1 systemd[1]: etc-machine\x2did.mount: Deactivated successfully. container-test-run-monitoring> machine1 # [1123634.550211] machine1 systemd[1]: Finished Permit User Sessions. container-test-run-monitoring> machine1 # [1123634.550924] machine1 systemd[1]: Started Console Getty. container-test-run-monitoring> machine1 # [1123634.550948] machine1 systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0 container-test-run-monitoring> machine1 # [1123634.550961] machine1 systemd[1]: Reached target Login Prompts. container-test-run-monitoring> machine2 # [1123634.360235] machine2 systemd[1]: Finished Import lastlog data into lastlog2 database. container-test-run-monitoring> machine2 # [1123634.516332] machine2 nsncd[226]: Aug 17 10:02:24.739 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" container-test-run-monitoring> machine2 # [1123634.516497] machine2 systemd[1]: Started Name Service Cache Daemon (nsncd). container-test-run-monitoring> machine2 # [1123634.516563] machine2 systemd[1]: Reached target Host and Network Name Lookups. container-test-run-monitoring> machine2 # [1123634.516620] machine2 systemd[1]: Reached target User and Group Name Lookups. container-test-run-monitoring> machine2 # [1123634.517771] machine2 systemd[1]: Starting User Login Management... container-test-run-monitoring> machine2 # [1123634.518735] machine2 systemd[1]: Starting Permit User Sessions... container-test-run-monitoring> machine2 # [1123634.544823] machine2 systemd[1]: etc-machine\x2did.mount: Deactivated successfully. container-test-run-monitoring> machine2 # [1123634.551013] machine2 systemd[1]: Finished Permit User Sessions. container-test-run-monitoring> machine2 # [1123634.551866] machine2 systemd[1]: Started Console Getty. container-test-run-monitoring> machine2 # [1123634.551898] machine2 systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0 container-test-run-monitoring> machine2 # [1123634.551913] machine2 systemd[1]: Reached target Login Prompts. container-test-run-monitoring> machine2 # [1123634.648463] machine2 dbus-broker-launch[227]: Looking up NSS user entry for 'systemd-timesync'... container-test-run-monitoring> machine2 # [1123634.649652] machine2 dbus-broker-launch[227]: NSS returned no entry for 'systemd-timesync' container-test-run-monitoring> machine2 # [1123634.649652] machine2 dbus-broker-launch[227]: Invalid user-name in /nix/store/ddhlan5vcqccpdhcb8jx6qg2j0jfarsm-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" container-test-run-monitoring> machine2 # [1123634.650650] machine2 systemd[1]: Started D-Bus System Message Bus. container-test-run-monitoring> machine2 # [1123634.659594] machine2 dbus-broker-launch[227]: Ready container-test-run-monitoring> machine1 # [1123634.648517] machine1 dbus-broker-launch[254]: Looking up NSS user entry for 'systemd-timesync'... container-test-run-monitoring> machine1 # [1123634.649199] machine1 dbus-broker-launch[254]: NSS returned no entry for 'systemd-timesync' container-test-run-monitoring> machine1 # [1123634.649199] machine1 dbus-broker-launch[254]: Invalid user-name in /nix/store/lsskh5r63qyicm496xsanzg21vjm63sm-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" container-test-run-monitoring> machine1 # [1123634.649680] machine1 systemd[1]: Started D-Bus System Message Bus. container-test-run-monitoring> machine1 # [1123634.659592] machine1 dbus-broker-launch[254]: Ready container-test-run-monitoring> machine1 # [1123634.743598] machine1 mimir[248]: ts=2026-08-17T10:02:24.870629559Z caller=main.go:232 level=info msg="Starting application" version="(version=3.1.4, branch=unknown, revision=unknown)" container-test-run-monitoring> machine1 # [1123634.743598] machine1 mimir[248]: ts=2026-08-17T10:02:24.873360979Z caller=server.go:401 level=info msg="server listening on addresses" http=[::]:3001 grpc=[::]:9095 container-test-run-monitoring> machine1 # [1123634.743598] machine1 mimir[248]: ts=2026-08-17T10:02:24.889117921Z caller=memberlist_client.go:541 level=info component=memberlist msg="Using memberlist cluster label and node name" cluster_label= node=machine1-6d12cc69 container-test-run-monitoring> machine1 # [1123634.743598] machine1 mimir[248]: ts=2026-08-17T10:02:24.89393165Z caller=modules.go:1054 level=info msg="The ruler is not being started because you need to configure the ruler storage." container-test-run-monitoring> machine1 # [1123634.743598] machine1 mimir[248]: ts=2026-08-17T10:02:24.896619061Z caller=inmemory.go:137 level=info msg="created in-memory index cache" maxItemSizeBytes=134217728 maxSizeBytes=1073741824 maxItems=maxInt container-test-run-monitoring> machine1 # [1123634.743598] machine1 mimir[248]: ts=2026-08-17T10:02:24.897271124Z caller=ingester.go:359 level=warn msg="-blocks-storage.backend=filesystem is for development and testing only; you should switch to an external object store for production use or use a shared filesystem" container-test-run-monitoring> machine1 # [1123634.743598] machine1 mimir[248]: ts=2026-08-17T10:02:24.898468408Z caller=ingester.go:513 level=info msg="TSDB idle compaction timeout set" timeout=1h2m0.935376347s container-test-run-monitoring> machine1 # [1123634.743598] machine1 mimir[248]: ts=2026-08-17T10:02:24.900777757Z caller=modules.go:1068 level=info msg="The ruler storage has not been configured. Not starting the ruler." container-test-run-monitoring> machine1 # [1123634.743598] machine1 mimir[248]: ts=2026-08-17T10:02:24.902143773Z caller=modules.go:936 level=info msg="The query-frontend has not been configured with the query-scheduler address. Because Mimir is running in monolithic mode, it's attempting an automatic frontend configuration. If queries are unresponsive, consider explicitly configuring the query-scheduler address for querier-frontend." address=127.0.0.1:9095 container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.016567603Z caller=modules.go:716 level=info msg="The querier worker has not been configured with the query-scheduler address. Because Mimir is running in monolithic mode, it's attempting an automatic worker configuration. If queries are unresponsive, consider explicitly configuring the query-scheduler address for querier worker." address=127.0.0.1:9095 container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.016618783Z caller=worker.go:148 level=info msg="Starting querier worker connected to query-scheduler" scheduler=127.0.0.1:9095 container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.017229685Z caller=module_service.go:82 level=info msg=starting module=sanity-check container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.017291946Z caller=sanity_check.go:32 level=info msg="Checking directories read/write access" container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.017219325Z caller=module_service.go:82 level=info msg=starting module=active-groups-cleanup-service container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.017220776Z caller=module_service.go:82 level=info msg=starting module=activity-tracker container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.017779478Z caller=sanity_check.go:37 level=info msg="Directories read/write access successfully checked" container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.017786168Z caller=sanity_check.go:39 level=info msg="Checking object storage config" container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.018030809Z caller=sanity_check.go:44 level=info msg="Object storage config successfully checked" container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.018053179Z caller=module_service.go:82 level=info msg=starting module=server container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.018150419Z caller=module_service.go:82 level=info msg=starting module=memberlist-kv container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.018626001Z caller=module_service.go:82 level=info msg=starting module=ingester-ring container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.018626081Z caller=module_service.go:82 level=info msg=starting module=querier-ring container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.018631111Z caller=module_service.go:82 level=info msg=starting module=store-queryable container-test-run-monitoring> machine1 # [1123634.843229] machine1 mimir[248]: ts=2026-08-17T10:02:25.018669881Z caller=module_service.go:82 level=info msg=starting module=querier-lifecycler container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.018787562Z caller=ring.go:390 level=info msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.018840882Z caller=basic_lifecycler.go:324 level=info msg="instance not found in the ring" instance=machine1 ring=querier container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.018874122Z caller=ring.go:390 level=info msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.018652381Z caller=module_service.go:82 level=info msg=starting module=query-scheduler container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.019109543Z caller=ring.go:390 level=info msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.019135403Z caller=module_service.go:82 level=info msg=starting module=ingester-service container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.019155953Z caller=module_service.go:82 level=info msg=starting module=compactor container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.018643121Z caller=module_service.go:82 level=info msg=starting module=store-gateway container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.019177043Z caller=module_service.go:82 level=info msg=starting module=distributor-service container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.019179793Z caller=module_service.go:82 level=info msg=starting module=query-frontend container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.019367824Z caller=basic_lifecycler.go:324 level=info msg="instance not found in the ring" instance=machine1 ring=store-gateway container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.019382534Z caller=basic_lifecycler_delegates.go:63 level=info msg="not loading tokens from file, tokens file path is empty" container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.019416974Z caller=frontend_scheduler_worker.go:146 level=info msg="adding connection to query-scheduler" addr=127.0.0.1:9095 container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.019727035Z caller=gateway.go:246 level=info msg="waiting until store-gateway is JOINING in the ring" container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.019781295Z caller=gateway.go:250 level=info msg="store-gateway is JOINING in the ring" container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.019796486Z caller=bucket_stores.go:215 level=info msg="synchronizing TSDB blocks for all users" container-test-run-monitoring> machine1 # [1123634.843556] machine1 mimir[248]: ts=2026-08-17T10:02:25.020185097Z caller=bucket_stores.go:228 level=info msg="successfully synchronized TSDB blocks for all users" container-test-run-monitoring> machine1 # [1123634.843889] machine1 mimir[248]: ts=2026-08-17T10:02:25.020455378Z caller=ring.go:390 level=info component=compactor msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [1123634.843889] machine1 mimir[248]: ts=2026-08-17T10:02:25.020464828Z caller=basic_lifecycler.go:324 level=info component=compactor msg="instance not found in the ring" instance=machine1 ring=compactor container-test-run-monitoring> machine1 # [1123634.843889] machine1 mimir[248]: ts=2026-08-17T10:02:25.020528749Z caller=basic_lifecycler.go:324 level=info msg="instance not found in the ring" instance=machine1 ring=distributor container-test-run-monitoring> machine1 # [1123634.843889] machine1 mimir[248]: ts=2026-08-17T10:02:25.020588269Z caller=distributor.go:882 level=info msg="waiting until distributor is ACTIVE in the ring" container-test-run-monitoring> machine1 # [1123634.843889] machine1 mimir[248]: ts=2026-08-17T10:02:25.020629179Z caller=module_service.go:82 level=info msg=starting module=querier container-test-run-monitoring> machine1 # [1123634.843889] machine1 mimir[248]: ts=2026-08-17T10:02:25.020642739Z caller=compactor.go:583 level=info component=compactor msg="waiting until compactor is ACTIVE in the ring" container-test-run-monitoring> machine1 # [1123634.843889] machine1 mimir[248]: ts=2026-08-17T10:02:25.020687239Z caller=worker.go:248 level=info msg="adding connection" addr=127.0.0.1:9095 in-use=true container-test-run-monitoring> machine1 # [1123634.843889] machine1 mimir[248]: ts=2026-08-17T10:02:25.026372072Z caller=ingester_tsdb.go:399 level=info msg="opening existing TSDBs" container-test-run-monitoring> machine1 # [1123634.843889] machine1 mimir[248]: ts=2026-08-17T10:02:25.028719371Z caller=lifecycler.go:694 level=info msg="not loading tokens from file, tokens file path is empty" container-test-run-monitoring> machine1 # [1123634.843889] machine1 mimir[248]: ts=2026-08-17T10:02:25.028783251Z caller=lifecycler.go:721 level=info msg="instance not found in ring, adding with no tokens" ring=ingester container-test-run-monitoring> machine1 # [1123634.843889] machine1 mimir[248]: ts=2026-08-17T10:02:25.028841461Z caller=lifecycler.go:562 level=info msg="auto-joining cluster after timeout" ring=ingester container-test-run-monitoring> machine1 # [1123634.879259] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.009753506Z caller=main.go:146 msg="Starting Loki" version="(version=3.7.6, branch=unknown, revision=unknown)" container-test-run-monitoring> machine1 # [1123634.879259] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.009783636Z caller=main.go:147 msg="Loading configuration file" filename=/nix/store/zhq0ryjwnyd6v5jh1hp6074m1gz0s337-validate-loki-conf container-test-run-monitoring> machine1 # [1123634.879259] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.009905887Z caller=modules.go:1597 msg="Ruler storage is not configured; ruler will not be started." config_key=ruler.storage container-test-run-monitoring> machine1 # [1123634.879259] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.012963099Z caller=server.go:397 msg="server listening on addresses" http=[::]:3002 grpc=[::]:9096 container-test-run-monitoring> machine1 # [1123634.879259] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.025681289Z caller=table_manager.go:136 index-store=tsdb-2025-11-01 msg="uploading tables" container-test-run-monitoring> machine1 # [1123634.879259] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.025758139Z caller=table_manager.go:300 index-store=tsdb-2025-11-01 msg="query readiness setup completed" duration=14.2µs distinct_users_len=0 distinct_users= container-test-run-monitoring> machine1 # [1123634.879259] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.025821389Z caller=shipper.go:165 index-store=tsdb-2025-11-01 msg="starting index shipper in RW mode" container-test-run-monitoring> machine1 # [1123634.879259] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.026352041Z caller=head_manager.go:313 index-store=tsdb-2025-11-01 component=tsdb-head-manager msg="loaded wals by period" groups=0 container-test-run-monitoring> machine1 # [1123634.879259] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.026452602Z caller=manager.go:86 index-store=tsdb-2025-11-01 component=tsdb-manager msg="loaded leftover local indices" err=null successful=true buckets=0 indices=0 failures=0 container-test-run-monitoring> machine1 # [1123634.879259] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.026469732Z caller=head_manager.go:313 index-store=tsdb-2025-11-01 component=tsdb-head-manager msg="loaded wals by period" groups=0 container-test-run-monitoring> machine1 # [1123634.879259] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.032957267Z caller=worker.go:134 component=querier msg="Starting querier worker using query-scheduler and scheduler ring for addresses" container-test-run-monitoring> machine1 # [1123634.879259] machine1 loki[247]: level=warn ts=2026-08-17T10:02:25.033221839Z caller=modules.go:1643 msg="RulerStorage is nil. Not starting the ruler." container-test-run-monitoring> machine1 # [1123634.879259] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.046771682Z caller=module_service.go:82 msg=starting module=cache-generation-loader container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.046812562Z caller=module_service.go:82 msg=starting module=server container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.046843792Z caller=module_service.go:82 msg=starting module=query-frontend-tripperware container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.046872192Z caller=module_service.go:82 msg=starting module=memberlist-kv container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.047701696Z caller=module_service.go:82 msg=starting module=query-scheduler-ring container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.047705316Z caller=module_service.go:82 msg=starting module=store container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.047706896Z caller=module_service.go:82 msg=starting module=ring container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048025967Z caller=ring.go:390 msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048048647Z caller=ring.go:390 msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048124237Z caller=module_service.go:82 msg=starting module=ingester container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048127717Z caller=module_service.go:82 msg=starting module=distributor container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048131587Z caller=module_service.go:82 msg=starting module=compactor container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048149887Z caller=module_service.go:82 msg=starting module=ingester-querier container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048178497Z caller=module_service.go:82 msg=starting module=rule-evaluator container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048225747Z caller=ring.go:390 component=distributor msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048253308Z caller=basic_lifecycler.go:324 msg="instance not found in the ring" instance=machine1 ring=compactor container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048225907Z caller=ingester.go:571 component=ingester msg="recovering from checkpoint" container-test-run-monitoring> machine1 # [1123634.880754] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.047986486Z caller=basic_lifecycler.go:324 msg="instance not found in the ring" instance=machine1 ring=scheduler container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048275108Z caller=basic_lifecycler.go:324 component=distributor msg="instance not found in the ring" instance=machine1 ring=distributor container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048282478Z caller=basic_lifecycler_delegates.go:63 msg="not loading tokens from file, tokens file path is empty" container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048284218Z caller=ring.go:390 msg="ring doesn't exist in KV store yet" container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=error ts=2026-08-17T10:02:25.048247258Z caller=ratestore.go:110 msg="error getting ingester clients" err="empty ring" container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048285078Z caller=basic_lifecycler_delegates.go:63 msg="not loading tokens from file, tokens file path is empty" container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.048289308Z caller=checkpoint.go:549 msg="tmp checkpoint cleanup starting" container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.04878182Z caller=compactor.go:420 msg="waiting until compactor is JOINING in the ring" container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.04880177Z caller=checkpoint.go:554 msg="tmp checkpoint cleanup done" duration=38.4µs success=true container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.04880469Z caller=ringmanager.go:186 msg="waiting until scheduler is JOINING in the ring" container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.04885747Z caller=recovery.go:45 component=ingester msg="no checkpoint found, treating as no-op" container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.050138535Z caller=ingester.go:587 component=ingester msg="recovered WAL checkpoint recovery finished" elapsed=1.932098ms errors=false container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.050155095Z caller=ingester.go:593 component=ingester msg="recovering from WAL" container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.050647237Z caller=ingester.go:609 component=ingester msg="WAL segment recovery finished" elapsed=2.41933ms errors=false container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.050712687Z caller=ingester.go:557 component=ingester msg="closing recoverer" container-test-run-monitoring> machine1 # [1123634.880960] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.050740167Z caller=ingester.go:565 component=ingester msg="WAL recovery finished" time=2.5337ms container-test-run-monitoring> machine1 # [1123634.881164] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.050857218Z caller=wal.go:173 msg=started component=wal container-test-run-monitoring> machine1 # [1123634.881164] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.050942608Z caller=lifecycler.go:694 component=ingester msg="not loading tokens from file, tokens file path is empty" container-test-run-monitoring> machine1 # [1123634.881164] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.050914208Z caller=ingester.go:778 component=ingester msg="sleeping for initial delay before starting periodic flushing" delay=3.266643689s container-test-run-monitoring> machine1 # [1123634.881164] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.051042769Z caller=lifecycler.go:721 component=ingester msg="instance not found in ring, adding with no tokens" ring=ingester container-test-run-monitoring> machine1 # [1123634.881164] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.051146349Z caller=lifecycler.go:562 component=ingester msg="auto-joining cluster after timeout" ring=ingester container-test-run-monitoring> machine1 # [1123634.942844] machine1 mimir[248]: ts=2026-08-17T10:02:25.163529721Z caller=compactor.go:588 level=info component=compactor msg="compactor is ACTIVE in the ring" container-test-run-monitoring> machine1 # [1123634.942844] machine1 mimir[248]: ts=2026-08-17T10:02:25.163717112Z caller=compactor.go:706 level=info component=compactor msg="discovering users from bucket" container-test-run-monitoring> machine1 # [1123634.942844] machine1 mimir[248]: ts=2026-08-17T10:02:25.163755822Z caller=compactor.go:716 level=info component=compactor msg="discovered users from bucket" users=0 container-test-run-monitoring> machine1 # [1123634.942844] machine1 mimir[248]: ts=2026-08-17T10:02:25.164086123Z caller=blocks_cleaner.go:241 level=info component=cleaner task=clean_up_users_during_startup msg="started blocks cleanup and maintenance" container-test-run-monitoring> machine1 # [1123634.942844] machine1 mimir[248]: ts=2026-08-17T10:02:25.164316134Z caller=blocks_cleaner.go:247 level=info component=cleaner task=clean_up_users_during_startup msg="successfully completed blocks cleanup and maintenance" container-test-run-monitoring> machine1 # [1123635.052995] machine1 postgresql-pre-start[402]: The files belonging to this database system will be owned by user "postgres". container-test-run-monitoring> machine1 # [1123635.052995] machine1 postgresql-pre-start[402]: This user must also own the server process. container-test-run-monitoring> machine1 # [1123635.053396] machine1 postgresql-pre-start[402]: The database cluster will be initialized with locale "en_US.UTF-8". container-test-run-monitoring> machine1 # [1123635.053396] machine1 postgresql-pre-start[402]: The default database encoding has accordingly been set to "UTF8". container-test-run-monitoring> machine1 # [1123635.053396] machine1 postgresql-pre-start[402]: The default text search configuration will be set to "english". container-test-run-monitoring> machine1 # [1123635.053396] machine1 postgresql-pre-start[402]: Data page checksums are enabled. container-test-run-monitoring> machine1 # [1123635.053396] machine1 postgresql-pre-start[402]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok container-test-run-monitoring> machine1 # [1123635.053963] machine1 postgresql-pre-start[402]: creating subdirectories ... ok container-test-run-monitoring> machine1 # [1123635.054109] machine1 postgresql-pre-start[402]: selecting dynamic shared memory implementation ... posix container-test-run-monitoring> machine1 # [1123635.054590] machine1 nginx-pre-start[400]: nginx: the configuration file /nix/store/fv4nnlwkn3q6xli6m4mq32l15b44mjqc-nginx.conf syntax is ok container-test-run-monitoring> machine1 # [1123635.054816] machine1 nginx-pre-start[400]: nginx: configuration file /nix/store/fv4nnlwkn3q6xli6m4mq32l15b44mjqc-nginx.conf test is successful container-test-run-monitoring> machine1 # [1123635.078856] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.213279347Z caller=ringmanager.go:190 msg="scheduler is JOINING in the ring" container-test-run-monitoring> machine1 # [1123635.078856] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.213470398Z caller=ringmanager.go:199 msg="waiting until scheduler is ACTIVE in the ring" container-test-run-monitoring> machine1 # [1123635.078856] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.23957288Z caller=compactor.go:424 msg="compactor is JOINING in the ring" container-test-run-monitoring> machine1 # [1123635.078856] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.239741061Z caller=compactor.go:434 msg="waiting until compactor is ACTIVE in the ring" container-test-run-monitoring> machine1 # [1123635.080111] machine1 postgresql-pre-start[402]: selecting default "max_connections" ... 100 container-test-run-monitoring> machine1 # [1123635.102672] machine1 postgresql-pre-start[402]: selecting default "shared_buffers" ... 128MB container-test-run-monitoring> machine1 # [1123635.116739] machine1 systemd-logind[269]: New seat seat0. container-test-run-monitoring> machine1 # [1123635.116856] machine1 systemd[1]: Started User Login Management. container-test-run-monitoring> machine1 # [1123635.117919] machine1 systemd[1]: Starting linger-users.service... container-test-run-monitoring> machine1 # [1123635.143474] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.356093099Z caller=ringmanager.go:203 msg="scheduler is ACTIVE in the ring" container-test-run-monitoring> machine1 # [1123635.143474] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.356195539Z caller=module_service.go:82 msg=starting module=query-scheduler container-test-run-monitoring> machine1 # [1123635.143474] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.35637646Z caller=module_service.go:82 msg=starting module=query-frontend container-test-run-monitoring> machine1 # [1123635.143474] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.35638094Z caller=module_service.go:82 msg=starting module=querier container-test-run-monitoring> machine1 # [1123635.143474] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.366181078Z caller=compactor.go:438 msg="compactor is ACTIVE in the ring" container-test-run-monitoring> machine1 # [1123635.143474] machine1 loki[247]: level=info ts=2026-08-17T10:02:25.366242279Z caller=loki.go:615 msg="Loki started" startup_time=430.840005ms container-test-run-monitoring> machine1 # [1123635.150693] machine1 alloy[245]: ts=2026-08-17T10:02:25.359589583Z level=info msg="Alloy is starting" container-test-run-monitoring> machine1 # [1123635.150693] machine1 alloy[245]: ts=2026-08-17T10:02:25.359878444Z level=info msg="boringcrypto enabled" enabled=false container-test-run-monitoring> machine1 # [1123635.150972] machine1 alloy[245]: ts=2026-08-17T10:02:25.363138827Z level=info msg="memory is not limited, skipping" package=github.com/KimMachineGun/automemlimit/memlimit container-test-run-monitoring> machine1 # [1123635.150972] machine1 alloy[245]: ts=2026-08-17T10:02:25.363230147Z level=info msg="no peer discovery configured: both join and discover peers are empty" service=cluster container-test-run-monitoring> machine1 # [1123635.150972] machine1 alloy[245]: ts=2026-08-17T10:02:25.366156429Z level=info msg="starting complete graph evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 container-test-run-monitoring> machine1 # [1123635.150972] machine1 alloy[245]: ts=2026-08-17T10:02:25.367028462Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=loki.relabel.journal duration=854.754µs container-test-run-monitoring> machine1 # [1123635.150972] machine1 alloy[245]: ts=2026-08-17T10:02:25.367036672Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=otel duration=2.57µs container-test-run-monitoring> machine1 # [1123635.150972] machine1 alloy[245]: ts=2026-08-17T10:02:25.367684715Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=loki.write.loki duration=643.533µs container-test-run-monitoring> machine1 # [1123635.150972] machine1 alloy[245]: ts=2026-08-17T10:02:25.367981276Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=loki.source.journal.all duration=288.312µs container-test-run-monitoring> machine1 # [1123635.150972] machine1 alloy[245]: ts=2026-08-17T10:02:25.369302311Z level=info msg="replaying WAL, this may take a while" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=wal dir=data-alloy/prometheus.remote_write.mimir/wal container-test-run-monitoring> machine1 # [1123635.150972] machine1 alloy[245]: ts=2026-08-17T10:02:25.369674742Z level=info msg="WAL segment loaded" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=wal segment=0 maxSegment=0 container-test-run-monitoring> machine1 # [1123635.150972] machine1 alloy[245]: ts=2026-08-17T10:02:25.370929357Z level=info msg="Starting WAL watcher" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=rw remote_name=fdeba4 url=http://machine1.clan/mimir/api/v1/push queue=fdeba4 container-test-run-monitoring> machine1 # [1123635.150972] machine1 alloy[245]: ts=2026-08-17T10:02:25.370969997Z level=info msg="Starting scraped metadata watcher" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=rw remote_name=fdeba4 url=http://machine1.clan/mimir/api/v1/push container-test-run-monitoring> machine1 # [1123635.151164] machine1 alloy[245]: ts=2026-08-17T10:02:25.371001448Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=prometheus.remote_write.mimir duration=3.016782ms container-test-run-monitoring> machine1 # [1123635.151164] machine1 alloy[245]: ts=2026-08-17T10:02:25.371172388Z level=info msg="Replaying WAL" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=rw remote_name=fdeba4 url=http://machine1.clan/mimir/api/v1/push queue=fdeba4 container-test-run-monitoring> machine1 # [1123635.151164] machine1 alloy[245]: ts=2026-08-17T10:02:25.371423429Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=prometheus.relabel.create_nixos_services_metric duration=415.641µs container-test-run-monitoring> machine1 # [1123635.151164] machine1 alloy[245]: ts=2026-08-17T10:02:25.37166624Z level=info msg="Parsed flag --collector.filesystem.mount-points-exclude" component_path=/ component_id=prometheus.exporter.unix.local_system collector=filesystem flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) container-test-run-monitoring> machine1 # [1123635.151164] machine1 alloy[245]: ts=2026-08-17T10:02:25.37166988Z level=info msg="Parsed flag --collector.filesystem.fs-types-exclude" component_path=/ component_id=prometheus.exporter.unix.local_system collector=filesystem flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ container-test-run-monitoring> machine1 # [1123635.151164] machine1 alloy[245]: ts=2026-08-17T10:02:25.3717607Z level=info msg="Parsed flag --collector.systemd.unit-include" component_path=/ component_id=prometheus.exporter.unix.local_system collector=systemd flag=.+ container-test-run-monitoring> machine1 # [1123635.151164] machine1 alloy[245]: ts=2026-08-17T10:02:25.37176694Z level=info msg="Parsed flag --collector.systemd.unit-exclude" component_path=/ component_id=prometheus.exporter.unix.local_system collector=systemd flag=.+\.(automount|device|mount|scope|slice) container-test-run-monitoring> machine1 # [1123635.151164] machine1 alloy[245]: ts=2026-08-17T10:02:25.371782721Z level=info msg="Enabled node_exporter collectors" component_path=/ component_id=prometheus.exporter.unix.local_system container-test-run-monitoring> machine1 # [1123635.151164] machine1 alloy[245]: ts=2026-08-17T10:02:25.371784121Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=cpu container-test-run-monitoring> machine1 # [1123635.150977] machine1 systemd[1]: linger-users.service: Deactivated successfully. container-test-run-monitoring> machine1 # [1123635.151372] machine1 alloy[245]: ts=2026-08-17T10:02:25.371784611Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=filesystem container-test-run-monitoring> machine1 # [1123635.151372] machine1 alloy[245]: ts=2026-08-17T10:02:25.371785081Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=meminfo container-test-run-monitoring> machine1 # [1123635.151372] machine1 alloy[245]: ts=2026-08-17T10:02:25.371785441Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=systemd container-test-run-monitoring> machine1 # [1123635.151372] machine1 alloy[245]: ts=2026-08-17T10:02:25.371796701Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=prometheus.exporter.unix.local_system duration=369.182µs container-test-run-monitoring> machine1 # [1123635.151372] machine1 alloy[245]: ts=2026-08-17T10:02:25.373379407Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=prometheus.scrape.scrape_metrics duration=1.578606ms container-test-run-monitoring> machine1 # [1123635.151372] machine1 alloy[245]: ts=2026-08-17T10:02:25.373404737Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=labelstore duration=20.14µs container-test-run-monitoring> machine1 # [1123635.151372] machine1 alloy[245]: ts=2026-08-17T10:02:25.373412227Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=tracing duration=5.49µs container-test-run-monitoring> machine1 # [1123635.151372] machine1 alloy[245]: ts=2026-08-17T10:02:25.373677918Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=logging duration=263.951µs container-test-run-monitoring> machine1 # [1123635.151372] machine1 alloy[245]: ts=2026-08-17T10:02:25.373725988Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=remotecfg duration=43.23µs container-test-run-monitoring> machine1 # [1123635.151372] machine1 alloy[245]: ts=2026-08-17T10:02:25.373747439Z level=info msg="applying non-TLS config to HTTP server" service=http container-test-run-monitoring> machine1 # [1123635.151372] machine1 alloy[245]: ts=2026-08-17T10:02:25.373763579Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=http duration=31.48µs container-test-run-monitoring> machine1 # [1123635.151262] machine1 systemd[1]: Finished linger-users.service. container-test-run-monitoring> machine1 # [1123635.151736] machine1 alloy[245]: ts=2026-08-17T10:02:25.373770639Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=cluster duration=2.1µs container-test-run-monitoring> machine1 # [1123635.151736] machine1 alloy[245]: ts=2026-08-17T10:02:25.373780139Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=livedebugging duration=4.351µs container-test-run-monitoring> machine1 # [1123635.151736] machine1 alloy[245]: ts=2026-08-17T10:02:25.373790819Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 node_id=ui duration=2.56µs container-test-run-monitoring> machine1 # [1123635.151736] machine1 alloy[245]: ts=2026-08-17T10:02:25.373795749Z level=info msg="finished complete graph evaluation" controller_path=/ controller_id="" trace_id=69f94b3406475b51ffbcdc99c77c5380 duration=8.117572ms container-test-run-monitoring> machine1 # [1123635.151736] machine1 alloy[245]: ts=2026-08-17T10:02:25.373895539Z level=info msg="scheduling loaded components and services" controller_id="" container-test-run-monitoring> machine1 # [1123635.151736] machine1 alloy[245]: ts=2026-08-17T10:02:25.374014329Z level=info msg="{^_^} Alloy is running" container-test-run-monitoring> machine1 # [1123635.151736] machine1 alloy[245]: ts=2026-08-17T10:02:25.37419116Z level=info msg="starting cluster node" service=cluster peers_count=0 peers="" advertise_addr=127.0.0.1:12345 minimum_cluster_size=0 minimum_size_wait_timeout=0s container-test-run-monitoring> machine1 # [1123635.151736] machine1 alloy[245]: ts=2026-08-17T10:02:25.374284661Z level=error msg="failed to register collector with remote server" service=remotecfg id=f9548bf2-633c-4970-b57d-02856568eb4c name="" err="noop client" container-test-run-monitoring> machine1 # [1123635.151736] machine1 alloy[245]: ts=2026-08-17T10:02:25.374498631Z level=info msg="peers changed" service=cluster peers_count=1 min_cluster_size=0 peers=machine1 container-test-run-monitoring> machine1 # [1123635.152172] machine1 alloy[245]: ts=2026-08-17T10:02:25.374982933Z level=info msg="now listening for http traffic" service=http addr=127.0.0.1:12345 container-test-run-monitoring> machine2 # [1123635.116280] machine2 systemd-logind[243]: New seat seat0. container-test-run-monitoring> machine2 # [1123635.116443] machine2 systemd[1]: Started User Login Management. container-test-run-monitoring> machine2 # [1123635.117553] machine2 systemd[1]: Starting linger-users.service... container-test-run-monitoring> machine2 # [1123635.151211] machine2 alloy[224]: ts=2026-08-17T10:02:25.359586173Z level=info msg="Alloy is starting" container-test-run-monitoring> machine2 # [1123635.151211] machine2 alloy[224]: ts=2026-08-17T10:02:25.359881064Z level=info msg="boringcrypto enabled" enabled=false container-test-run-monitoring> machine2 # [1123635.151535] machine2 alloy[224]: ts=2026-08-17T10:02:25.363533378Z level=info msg="memory is not limited, skipping" package=github.com/KimMachineGun/automemlimit/memlimit container-test-run-monitoring> machine2 # [1123635.151535] machine2 alloy[224]: ts=2026-08-17T10:02:25.363574028Z level=info msg="no peer discovery configured: both join and discover peers are empty" service=cluster container-test-run-monitoring> machine2 # [1123635.151535] machine2 alloy[224]: ts=2026-08-17T10:02:25.366154038Z level=info msg="starting complete graph evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 container-test-run-monitoring> machine2 # [1123635.151535] machine2 alloy[224]: ts=2026-08-17T10:02:25.367684744Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=loki.write.loki duration=1.510535ms container-test-run-monitoring> machine2 # [1123635.151535] machine2 alloy[224]: ts=2026-08-17T10:02:25.367960296Z level=info msg="Parsed flag --collector.filesystem.mount-points-exclude" component_path=/ component_id=prometheus.exporter.unix.local_system collector=filesystem flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) container-test-run-monitoring> machine2 # [1123635.151535] machine2 alloy[224]: ts=2026-08-17T10:02:25.367987866Z level=info msg="Parsed flag --collector.filesystem.fs-types-exclude" component_path=/ component_id=prometheus.exporter.unix.local_system collector=filesystem flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ container-test-run-monitoring> machine2 # [1123635.151535] machine2 alloy[224]: ts=2026-08-17T10:02:25.368234817Z level=info msg="Parsed flag --collector.systemd.unit-include" component_path=/ component_id=prometheus.exporter.unix.local_system collector=systemd flag=.+ container-test-run-monitoring> machine2 # [1123635.151535] machine2 alloy[224]: ts=2026-08-17T10:02:25.368241967Z level=info msg="Parsed flag --collector.systemd.unit-exclude" component_path=/ component_id=prometheus.exporter.unix.local_system collector=systemd flag=.+\.(automount|device|mount|scope|slice) container-test-run-monitoring> machine2 # [1123635.151535] machine2 alloy[224]: ts=2026-08-17T10:02:25.368493878Z level=info msg="Enabled node_exporter collectors" component_path=/ component_id=prometheus.exporter.unix.local_system container-test-run-monitoring> machine2 # [1123635.151535] machine2 alloy[224]: ts=2026-08-17T10:02:25.368495948Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=cpu container-test-run-monitoring> machine2 # [1123635.151692] machine2 alloy[224]: ts=2026-08-17T10:02:25.368507628Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=filesystem container-test-run-monitoring> machine2 # [1123635.151692] machine2 alloy[224]: ts=2026-08-17T10:02:25.368508218Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=meminfo container-test-run-monitoring> machine2 # [1123635.151692] machine2 alloy[224]: ts=2026-08-17T10:02:25.368508828Z level=info msg=collector component_path=/ component_id=prometheus.exporter.unix.local_system name=systemd container-test-run-monitoring> machine2 # [1123635.151692] machine2 alloy[224]: ts=2026-08-17T10:02:25.368545758Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=prometheus.exporter.unix.local_system duration=854.253µs container-test-run-monitoring> machine2 # [1123635.151692] machine2 alloy[224]: ts=2026-08-17T10:02:25.368559078Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=tracing duration=7.86µs container-test-run-monitoring> machine2 # [1123635.151692] machine2 alloy[224]: ts=2026-08-17T10:02:25.369883873Z level=info msg="replaying WAL, this may take a while" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=wal dir=data-alloy/prometheus.remote_write.mimir/wal container-test-run-monitoring> machine2 # [1123635.151692] machine2 alloy[224]: ts=2026-08-17T10:02:25.370026794Z level=info msg="WAL segment loaded" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=wal segment=0 maxSegment=0 container-test-run-monitoring> machine2 # [1123635.151692] machine2 alloy[224]: ts=2026-08-17T10:02:25.370947757Z level=info msg="Starting WAL watcher" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=rw remote_name=95eacb url=http://machine1.clan/mimir/api/v1/push queue=95eacb container-test-run-monitoring> machine2 # [1123635.151692] machine2 alloy[224]: ts=2026-08-17T10:02:25.370973708Z level=info msg="Starting scraped metadata watcher" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=rw remote_name=95eacb url=http://machine1.clan/mimir/api/v1/push container-test-run-monitoring> machine2 # [1123635.151692] machine2 alloy[224]: ts=2026-08-17T10:02:25.371004708Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=prometheus.remote_write.mimir duration=2.442989ms container-test-run-monitoring> machine2 # [1123635.151862] machine2 alloy[224]: ts=2026-08-17T10:02:25.371193968Z level=info msg="Replaying WAL" component_path=/ component_id=prometheus.remote_write.mimir subcomponent=rw remote_name=95eacb url=http://machine1.clan/mimir/api/v1/push queue=95eacb container-test-run-monitoring> machine2 # [1123635.151862] machine2 alloy[224]: ts=2026-08-17T10:02:25.371426499Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=prometheus.relabel.create_nixos_services_metric duration=415.792µs container-test-run-monitoring> machine2 # [1123635.151862] machine2 alloy[224]: ts=2026-08-17T10:02:25.373278906Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=prometheus.scrape.scrape_metrics duration=1.848248ms container-test-run-monitoring> machine2 # [1123635.151862] machine2 alloy[224]: ts=2026-08-17T10:02:25.373462557Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=remotecfg duration=177.871µs container-test-run-monitoring> machine2 # [1123635.151862] machine2 alloy[224]: ts=2026-08-17T10:02:25.373507867Z level=info msg="applying non-TLS config to HTTP server" service=http container-test-run-monitoring> machine2 # [1123635.151862] machine2 alloy[224]: ts=2026-08-17T10:02:25.373509007Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=http duration=43.441µs container-test-run-monitoring> machine2 # [1123635.151862] machine2 alloy[224]: ts=2026-08-17T10:02:25.373515578Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=cluster duration=2.751µs container-test-run-monitoring> machine2 # [1123635.151862] machine2 alloy[224]: ts=2026-08-17T10:02:25.373520387Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=otel duration=2.251µs container-test-run-monitoring> machine2 # [1123635.151862] machine2 alloy[224]: ts=2026-08-17T10:02:25.373529627Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=livedebugging duration=7.34µs container-test-run-monitoring> machine2 # [1123635.151862] machine2 alloy[224]: ts=2026-08-17T10:02:25.373535007Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=ui duration=3.1µs container-test-run-monitoring> machine2 # [1123635.151841] machine2 systemd[1]: linger-users.service: Deactivated successfully. container-test-run-monitoring> machine2 # [1123635.152093] machine2 alloy[224]: ts=2026-08-17T10:02:25.373745698Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=loki.relabel.journal duration=208.54µs container-test-run-monitoring> machine2 # [1123635.152093] machine2 alloy[224]: ts=2026-08-17T10:02:25.373941789Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=loki.source.journal.all duration=192.331µs container-test-run-monitoring> machine2 # [1123635.152093] machine2 alloy[224]: ts=2026-08-17T10:02:25.373952729Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=labelstore duration=6.73µs container-test-run-monitoring> machine2 # [1123635.152093] machine2 alloy[224]: ts=2026-08-17T10:02:25.37423806Z level=info msg="finished node evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 node_id=logging duration=282.571µs container-test-run-monitoring> machine2 # [1123635.152093] machine2 alloy[224]: ts=2026-08-17T10:02:25.37424367Z level=info msg="finished complete graph evaluation" controller_path=/ controller_id="" trace_id=0901c83fcf31430f31cb25d06e327d96 duration=8.568773ms container-test-run-monitoring> machine2 # [1123635.152093] machine2 alloy[224]: ts=2026-08-17T10:02:25.374306301Z level=info msg="scheduling loaded components and services" controller_id="" container-test-run-monitoring> machine2 # [1123635.152093] machine2 alloy[224]: ts=2026-08-17T10:02:25.374401821Z level=info msg="{^_^} Alloy is running" container-test-run-monitoring> machine2 # [1123635.152093] machine2 alloy[224]: ts=2026-08-17T10:02:25.374459051Z level=info msg="starting cluster node" service=cluster peers_count=0 peers="" advertise_addr=127.0.0.1:12345 minimum_cluster_size=0 minimum_size_wait_timeout=0s container-test-run-monitoring> machine2 # [1123635.152093] machine2 alloy[224]: ts=2026-08-17T10:02:25.374533422Z level=info msg="peers changed" service=cluster peers_count=1 min_cluster_size=0 peers=machine2 container-test-run-monitoring> machine2 # [1123635.152093] machine2 alloy[224]: ts=2026-08-17T10:02:25.374601652Z level=error msg="failed to register collector with remote server" service=remotecfg id=aadb5b63-676e-42f8-a09e-211410686bda name="" err="noop client" container-test-run-monitoring> machine2 # [1123635.151924] machine2 systemd[1]: Finished linger-users.service. container-test-run-monitoring> machine2 # [1123635.152317] machine2 systemd[1]: Reached target Multi-User System. container-test-run-monitoring> machine2 # [1123635.152459] machine2 alloy[224]: ts=2026-08-17T10:02:25.375190014Z level=info msg="now listening for http traffic" service=http addr=127.0.0.1:12345 container-test-run-monitoring> machine2 # [1123635.152536] machine2 systemd[1]: Startup finished in 1.930s. container-test-run-monitoring> machine2 # [1123635.445097] machine2 systemd-networkd[214]: eth1: Gained IPv6LL container-test-run-monitoring> machine1 # [1123635.460213] machine1 postgresql-pre-start[402]: selecting default time zone ... UTC container-test-run-monitoring> machine1 # [1123635.460816] machine1 postgresql-pre-start[402]: creating configuration files ... ok container-test-run-monitoring> machine1 # [1123635.510610] machine1 systemd-networkd[235]: eth1: Gained IPv6LL container-test-run-monitoring> machine1 # [1123635.566976] machine1 postgresql-pre-start[402]: running bootstrap script ... ok container-test-run-monitoring> machine1 # [1123635.585345] machine1 systemd[1]: Started Nginx Web Server. container-test-run-monitoring> machine1 # [1123635.842789] machine1 mimir[248]: ts=2026-08-17T10:02:26.021240086Z caller=gateway.go:286 level=info msg="waiting until store-gateway is ACTIVE in the ring" container-test-run-monitoring> machine1 # [1123635.891273] machine1 postgresql-pre-start[402]: performing post-bootstrap initialization ... ok container-test-run-monitoring> machine1 # [1123636.025820] machine1 postgresql-pre-start[402]: syncing data to disk ... ok container-test-run-monitoring> machine1 # [1123636.025820] machine1 postgresql-pre-start[402]: initdb: warning: enabling "trust" authentication for local connections container-test-run-monitoring> machine1 # [1123636.025820] machine1 postgresql-pre-start[402]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. container-test-run-monitoring> machine1 # [1123636.025820] machine1 postgresql-pre-start[402]: Success. You can now start the database server using: container-test-run-monitoring> machine1 # [1123636.025820] machine1 postgresql-pre-start[402]: pg_ctl -D /var/lib/postgresql/18 -l logfile start container-test-run-monitoring> machine1 # [1123636.043609] machine1 mimir[248]: ts=2026-08-17T10:02:26.21529359Z caller=gateway.go:290 level=info msg="store-gateway is ACTIVE in the ring" container-test-run-monitoring> machine1 # [1123636.043609] machine1 mimir[248]: ts=2026-08-17T10:02:26.21534379Z caller=mimir.go:1103 level=info msg="Application started" container-test-run-monitoring> machine1 # [1123636.680442] machine1 postgres[427]: [427] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit container-test-run-monitoring> machine1 # [1123636.687372] machine1 postgres[427]: [427] LOG: listening on IPv6 address "::1", port 5432 container-test-run-monitoring> machine1 # [1123636.687372] machine1 postgres[427]: [427] LOG: listening on IPv4 address "127.0.0.1", port 5432 container-test-run-monitoring> machine1 # [1123636.688086] machine1 postgres[427]: [427] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" container-test-run-monitoring> machine1 # [1123636.691237] machine1 postgres[438]: [438] LOG: database system was shut down at 2026-08-17 10:02:26 GMT container-test-run-monitoring> machine1 # [1123636.693647] machine1 postgres[427]: [427] LOG: database system is ready to accept connections container-test-run-monitoring> machine1 # [1123637.336342] machine1 postgresql-post-start[451]: CREATE ROLE container-test-run-monitoring> machine1 # [1123637.405768] machine1 postgresql-post-start[456]: CREATE DATABASE container-test-run-monitoring> machine1 # [1123637.413451] machine1 systemd[1]: Started PostgreSQL Server. container-test-run-monitoring> machine1 # [1123637.414393] machine1 systemd[1]: Starting PostgreSQL Setup Scripts... container-test-run-monitoring> machine1 # [1123637.457496] machine1 systemd[1]: Finished PostgreSQL Setup Scripts. container-test-run-monitoring> machine1 # [1123637.457770] machine1 systemd[1]: Reached target PostgreSQL. container-test-run-monitoring> machine1 # [1123637.458732] machine1 systemd[1]: Starting Grafana Service Daemon... container-test-run-monitoring> machine1 # [1123638.178750] machine1 loki[247]: level=info ts=2026-08-17T10:02:28.357266918Z caller=scheduler.go:658 msg="this scheduler is in the ReplicationSet, will now accept requests." container-test-run-monitoring> machine1 # [1123638.178750] machine1 loki[247]: level=info ts=2026-08-17T10:02:28.357333199Z caller=worker.go:235 component=querier msg="adding connection" addr=127.0.0.1:9096 container-test-run-monitoring> machine1 # [1123640.024180] machine1 systemd[1]: Started Grafana Service Daemon. container-test-run-monitoring> machine1 # [1123640.024369] machine1 systemd[1]: Reached target Multi-User System. container-test-run-monitoring> machine1 # [1123640.024491] machine1 systemd[1]: Startup finished in 6.778s. container-test-run-monitoring> machine1 # [1123640.179044] machine1 loki[247]: level=info ts=2026-08-17T10:02:30.366778626Z caller=compactor.go:497 msg="this instance has been chosen to run the compactor, starting compactor" container-test-run-monitoring> machine1 # [1123640.179044] machine1 loki[247]: level=info ts=2026-08-17T10:02:30.366880866Z caller=tables_manager.go:70 msg="waiting 10m0s for ring to stay stable and previous compactions to finish before starting compactor" container-test-run-monitoring> machine1: (finished: waiting for unit multi-user.target, in 7.66 seconds) container-test-run-monitoring> machine2: waiting for unit multi-user.target container-test-run-monitoring> machine2: (finished: waiting for unit multi-user.target, in 0.01 seconds) container-test-run-monitoring> machine1: waiting for unit alloy container-test-run-monitoring> machine1: (finished: waiting for unit alloy, in 0.01 seconds) container-test-run-monitoring> machine2: waiting for unit alloy container-test-run-monitoring> machine2: (finished: waiting for unit alloy, in 0.01 seconds) container-test-run-monitoring> machine1: must succeed: test -f /etc/alloy/config.alloy container-test-run-monitoring> machine1: (finished: must succeed: test -f /etc/alloy/config.alloy, in 0.01 seconds) container-test-run-monitoring> machine2: must succeed: test -f /etc/alloy/config.alloy container-test-run-monitoring> machine2: (finished: must succeed: test -f /etc/alloy/config.alloy, in 0.01 seconds) container-test-run-monitoring> machine2: must succeed: test -f /etc/alloy/local-extension.alloy container-test-run-monitoring> machine2: (finished: must succeed: test -f /etc/alloy/local-extension.alloy, in 0.01 seconds) container-test-run-monitoring> machine1: must succeed: systemctl show alloy --property=ExecStart | grep -F '/etc/alloy' container-test-run-monitoring> machine1: (finished: must succeed: systemctl show alloy --property=ExecStart | grep -F '/etc/alloy', in 0.02 seconds) container-test-run-monitoring> machine2: must succeed: systemctl show alloy --property=ExecStart | grep -F '/etc/alloy' container-test-run-monitoring> machine2: (finished: must succeed: systemctl show alloy --property=ExecStart | grep -F '/etc/alloy', in 0.02 seconds) container-test-run-monitoring> machine1: waiting for unit loki container-test-run-monitoring> machine1: (finished: waiting for unit loki, in 0.01 seconds) container-test-run-monitoring> machine1: waiting for unit mimir container-test-run-monitoring> machine1: (finished: waiting for unit mimir, in 0.01 seconds) container-test-run-monitoring> machine1: waiting for unit grafana container-test-run-monitoring> machine1: (finished: waiting for unit grafana, in 0.01 seconds) container-test-run-monitoring> machine2: must succeed: test "$(grep -c '^loki.source.journal ' /etc/alloy/config.alloy)" = 1 container-test-run-monitoring> machine2: (finished: must succeed: test "$(grep -c '^loki.source.journal ' /etc/alloy/config.alloy)" = 1, in 0.01 seconds) container-test-run-monitoring> machine2: must succeed: cat /etc/alloy/config.alloy container-test-run-monitoring> machine2: (finished: must succeed: cat /etc/alloy/config.alloy, in 0.03 seconds) container-test-run-monitoring> (finished: run the VM test script, in 7.83 seconds) container-test-run-monitoring> test script finished in 7.89s container-test-run-monitoring> cleanup container-test-run-monitoring> kill NspawnMachine (pid 52) container-test-run-monitoring> kill NspawnMachine (pid 53) container-test-run-monitoring> Container machine1 terminated by signal KILL. container-test-run-monitoring> Container machine2 terminated by signal KILL. container-test-run-monitoring> (finished: cleanup, in 0.28 seconds) post-build step Upload to niks3: ok time=2026-08-17T10:02:31.668Z level=INFO msg="Uploading 0 paths to niks3.clan.lol (1 already cached)" time=2026-08-17T10:02:32.524Z level=WARN msg="Request returned retryable status, retrying" attempt=1 max_attempts=6 backoff=100ms status=500 url="https://s3.eu-central-003.backblazeb2.com/clan-cache-geninf/4lnw2sa75r9pbw28cqkijw75d9mvc6nh.ls?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=003eaae730241090000000009%2F20260817%2Feu-central-003%2Fs3%2Faws4_request&X-Amz-Date=20260817T100231Z&X-Amz-Expires=18000&X-Amz-SignedHeaders=host&X-Amz-Signature=681e14288b4171c2bb52d478441e4bc089da6c7d36c45cd205808e9f9e112336" time=2026-08-17T10:02:32.862Z level=INFO msg="Uploading 1 narinfos" time=2026-08-17T10:02:33.784Z level=INFO msg="Upload complete. (2.238s)"