build01: building gfhd6xchkyfzvg7w64kbanf9zy7mgpmz-container-test-run-dm-dns.drv
Machine state will be reset. To keep it, pass --keep-machine-state
start all VLans
(finished: start all VLans, in 0.00 seconds)
Initializing machine ID from random generator.
Test will time out and terminate in 3600.0 seconds
run the VM test script
additionally exposed symbols:
    client, server,
    vlan1,
    start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh
start all VMs
client: systemd-nspawn running (pid 52)
server: systemd-nspawn running (pid 53)
client: Waiting for journal at /build/vm-state-client/var/log/journal...
server: Waiting for journal at /build/vm-state-server/var/log/journal...
(finished: start all VMs, in 0.00 seconds)
server: waiting for unit unbound.service
nixos-nspawn(client): TAP vde-tap1 not found; container will be isolated from VDE
nixos-nspawn(client): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths.
nixos-nspawn(server): TAP vde-tap1 not found; container will be isolated from VDE
nixos-nspawn(server): A common reason for this is that /dev/net is not available in the Nix sandbox. Try adding /dev/net to extra-sandbox-paths.
Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file.
░ Spawning container client on /build/vm-state-client.
Note: in a future version of systemd-nspawn the default set of permitted socket address families will be restricted to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= to configure the set of permitted socket address families, or set RestrictAddressFamilies= in a .nspawn file.
░ Spawning container server on /build/vm-state-server.
server # [348139.427809] server systemd-journald[100]: Journal started
server # [348139.427870] server systemd-journald[100]: Runtime Journal (/run/log/journal/8346f3467bc9418496d965e57e43c405) is 8M, max 2.5G, 2.4G free.
server # [348139.436461] server systemd[1]: Finished Apply Kernel Variables.
server # [348139.449880] server systemd[1]: Finished Create Static Device Nodes in /dev gracefully.
server # [348139.470218] server systemd[1]: Starting Flush Journal to Persistent Storage...
server # [348139.471366] server systemd[1]: Starting Network Name Resolution...
server # [348139.472064] server systemd[1]: Starting Create Static Device Nodes in /dev...
server # [348139.481241] server systemd-journald[100]: Time spent on flushing to /var/log/journal/8346f3467bc9418496d965e57e43c405 is 1.665ms for 7 entries.
server # [348139.481241] server systemd-journald[100]: System Journal (/var/log/journal/8346f3467bc9418496d965e57e43c405) is 8M, max 4G, 3.9G free.
server # [348139.504975] server systemd[1]: Finished Flush Journal to Persistent Storage.
server # [348139.508285] server systemd[1]: Finished Create Static Device Nodes in /dev.
server # [348139.509394] server systemd[1]: Reached target Preparation for Local File Systems.
server # [348139.509502] server systemd[1]: Reached target Local File Systems.
server # [348139.510199] server systemd[1]: Listening on Boot Loader Control Service Socket.
server # [348139.510251] server systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container
server # [348139.511087] server systemd[1]: Starting Save Transient machine-id to Disk...
server # [348139.511799] server systemd[1]: Starting Create System Files and Directories...
server # [348139.511835] server systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys
server # [348139.527839] server systemd-tmpfiles[165]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted
server # [348139.527994] server systemd-tmpfiles[165]: fchmod() of /var/log/journal failed: Operation not permitted
server # [348139.528114] server systemd-tmpfiles[165]: fchmod() of /var/log/journal/8346f3467bc9418496d965e57e43c405 failed: Operation not permitted
server # [348139.528265] server systemd-tmpfiles[165]: fchmod() of /run/log/journal failed: Operation not permitted
server # [348139.529686] server systemd[1]: Finished Create System Files and Directories.
server # [348139.530797] server systemd[1]: Starting Rebuild Journal Catalog...
server # [348139.531568] server systemd[1]: Starting Record System Boot/Shutdown in UTMP...
client # [348139.421163] client systemd-journald[91]: Journal started
client # [348139.421214] client systemd-journald[91]: Runtime Journal (/run/log/journal/dd47e8fcf6ab42e69bbf88a2fd0b5e73) is 8M, max 2.5G, 2.4G free.
client # [348139.436877] client systemd[1]: Finished Apply Kernel Variables.
client # [348139.450828] client systemd[1]: Finished Create Static Device Nodes in /dev gracefully.
client # [348139.469101] client systemd[1]: Starting Flush Journal to Persistent Storage...
client # [348139.470097] client systemd[1]: Starting Network Name Resolution...
client # [348139.470808] client systemd[1]: Starting Create Static Device Nodes in /dev...
client # [348139.481270] client systemd-journald[91]: Time spent on flushing to /var/log/journal/dd47e8fcf6ab42e69bbf88a2fd0b5e73 is 1.835ms for 7 entries.
client # [348139.481270] client systemd-journald[91]: System Journal (/var/log/journal/dd47e8fcf6ab42e69bbf88a2fd0b5e73) is 8M, max 4G, 3.9G free.
client # [348139.504874] client systemd[1]: Finished Flush Journal to Persistent Storage.
client # [348139.508327] client systemd[1]: Finished Create Static Device Nodes in /dev.
client # [348139.509517] client systemd[1]: Reached target Preparation for Local File Systems.
client # [348139.509617] client systemd[1]: Reached target Local File Systems.
client # [348139.510588] client systemd[1]: Listening on Boot Loader Control Service Socket.
client # [348139.510636] client systemd[1]: Update Boot Loader Random Seed skipped, unmet condition check ConditionVirtualization=!container
client # [348139.511469] client systemd[1]: Starting Save Transient machine-id to Disk...
client # [348139.512290] client systemd[1]: Starting Create System Files and Directories...
client # [348139.512327] client systemd[1]: Rule-based Manager for Device Events and Files skipped, unmet condition check ConditionPathIsReadWrite=/sys
client # [348139.531113] client systemd-tmpfiles[155]: Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted
client # [348139.531313] client systemd-tmpfiles[155]: fchmod() of /var/log/journal failed: Operation not permitted
client # [348139.531446] client systemd-tmpfiles[155]: fchmod() of /var/log/journal/dd47e8fcf6ab42e69bbf88a2fd0b5e73 failed: Operation not permitted
client # [348139.531647] client systemd-tmpfiles[155]: fchmod() of /run/log/journal failed: Operation not permitted
client # [348139.533252] client systemd[1]: Finished Create System Files and Directories.
client # [348139.534352] client systemd[1]: Starting Rebuild Journal Catalog...
client # [348139.535111] client systemd[1]: Starting Record System Boot/Shutdown in UTMP...
client # [348139.547841] client systemd[1]: Finished Record System Boot/Shutdown in UTMP.
client # [348139.554479] client systemd[1]: Finished Save Transient machine-id to Disk.
client # [348139.554771] client systemd[1]: Finished Rebuild Journal Catalog.
client # [348139.556542] client systemd[1]: Starting Update is Completed...
client # [348139.569663] client systemd[1]: Finished Update is Completed.
client # [348139.606924] client systemd[1]: Finished Firewall.
client # [348139.607114] client systemd[1]: Reached target Preparation for Network.
client # [348139.607366] client systemd[1]: Listening on Network Management Resolve Hook Socket.
client # [348139.608496] client systemd[1]: Starting Network Management...
server # [348139.545856] server systemd[1]: Finished Record System Boot/Shutdown in UTMP.
server # [348139.551332] server systemd[1]: Finished Rebuild Journal Catalog.
server # [348139.552777] server systemd[1]: Starting Update is Completed...
server # [348139.554638] server systemd[1]: Finished Save Transient machine-id to Disk.
server # [348139.563827] server systemd[1]: Finished Update is Completed.
server # [348139.602505] server systemd[1]: Finished Firewall.
server # [348139.602701] server systemd[1]: Reached target Preparation for Network.
server # [348139.602981] server systemd[1]: Listening on Network Management Resolve Hook Socket.
server # [348139.604618] server systemd[1]: Starting Network Management...
server # [348140.436109] server systemd[1]: etc-machine\x2did.mount: Deactivated successfully.
server # [348140.438585] server systemd-networkd[218]: Failed to increase receive buffer size for general netlink socket, ignoring: Operation not permitted
server # [348140.438682] server systemd-networkd[218]: Failed to increase receive buffer size for nftables netlink socket, ignoring: Operation not permitted
server # [348140.448089] server systemd-networkd[218]: /etc/systemd/network/99-ethernet-default-dhcp.network: No valid settings found in the [Match] section, ignoring file. To match all interfaces, add Name=* in the [Match] section.
server # [348140.448254] server systemd-networkd[218]: /etc/systemd/network/99-wireless-client-dhcp.network: No valid settings found in the [Match] section, ignoring file. To match all interfaces, add Name=* in the [Match] section.
server # [348140.448420] server systemd-networkd[218]: lo: Link UP
server # [348140.448425] server systemd-networkd[218]: lo: Gained carrier
server # [348140.448609] server systemd-networkd[218]: eth1: Configuring with /etc/systemd/network/40-eth1.network.
server # [348140.449124] server systemd-networkd[218]: eth1: Link UP
server # [348140.449298] server systemd[1]: Started Network Management.
server # [348140.449453] server systemd-networkd[218]: eth1: Gained carrier
server # [348140.451021] server systemd[1]: Starting Enable Persistent Storage in systemd-networkd...
server # [348140.482210] server systemd[1]: Finished Enable Persistent Storage in systemd-networkd.
client # [348140.435045] client systemd[1]: etc-machine\x2did.mount: Deactivated successfully.
client # [348140.484620] client systemd-networkd[209]: Failed to increase receive buffer size for general netlink socket, ignoring: Operation not permitted
client # [348140.484717] client systemd-networkd[209]: Failed to increase receive buffer size for nftables netlink socket, ignoring: Operation not permitted
client # [348140.499431] client systemd-networkd[209]: /etc/systemd/network/99-ethernet-default-dhcp.network: No valid settings found in the [Match] section, ignoring file. To match all interfaces, add Name=* in the [Match] section.
client # [348140.499604] client systemd-networkd[209]: /etc/systemd/network/99-wireless-client-dhcp.network: No valid settings found in the [Match] section, ignoring file. To match all interfaces, add Name=* in the [Match] section.
client # [348140.499778] client systemd-networkd[209]: lo: Link UP
client # [348140.499783] client systemd-networkd[209]: lo: Gained carrier
client # [348140.499982] client systemd-networkd[209]: eth1: Configuring with /etc/systemd/network/40-eth1.network.
client # [348140.500494] client systemd-networkd[209]: eth1: Link UP
client # [348140.500658] client systemd[1]: Started Network Management.
client # [348140.500801] client systemd-networkd[209]: eth1: Gained carrier
client # [348140.502266] client systemd[1]: Starting Enable Persistent Storage in systemd-networkd...
client # [348140.542663] client systemd[1]: Finished Enable Persistent Storage in systemd-networkd.
client # [348140.788754] client systemd-resolved[130]: Positive Trust Anchors:
client # [348140.788771] client systemd-resolved[130]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
client # [348140.788775] client systemd-resolved[130]: . IN DS 38696 8 2 683d2d0acb8c9b712a1948b27f741219298d0a450d612c483af444a4c0fb2b16
client # [348140.788810] client systemd-resolved[130]: Negative trust anchors: home.arpa 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 170.0.0.192.in-addr.arpa 171.0.0.192.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa ipv4only.arpa resolver.arpa corp home internal intranet lan local private test
client # [348140.789536] client systemd-resolved[130]: Using system hostname 'client'.
client # [348140.791308] client systemd[1]: Started Network Name Resolution.
client # [348140.791446] client systemd[1]: Reached target Network.
client # [348140.791574] client systemd[1]: Reached target System Initialization.
client # [348140.791757] client systemd[1]: Started Watch for zone file changes.
client # [348140.791820] client systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container
client # [348140.791868] client systemd[1]: Started Daily Cleanup of Temporary Directories.
client # [348140.791906] client systemd[1]: Reached target Path Units.
client # [348140.791975] client systemd[1]: Reached target Timer Units.
client # [348140.792231] client systemd[1]: Listening on D-Bus System Message Bus Socket.
client # [348140.792519] client systemd[1]: Listening on Nix Daemon Socket.
client # [348140.792757] client systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.
client # [348140.792807] client systemd[1]: Reached target Socket Units.
client # [348140.792898] client systemd[1]: Reached target Basic System.
client # [348140.795100] client systemd[1]: Starting data mesher daemon...
client # [348140.796422] client systemd[1]: Starting Import lastlog data into lastlog2 database...
client # [348140.797864] client systemd[1]: Starting Name Service Cache Daemon (nsncd)...
client # [348140.800032] client systemd[1]: Starting D-Bus System Message Bus...
client # [348140.819072] client systemd[1]: Finished Import lastlog data into lastlog2 database.
client # [348140.975037] client systemd[1]: Started Name Service Cache Daemon (nsncd).
client # [348140.975124] client systemd[1]: Reached target User and Group Name Lookups.
client # [348140.975266] client nsncd[216]: Oct 20 10:05:45.947 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"
client # [348140.977404] client systemd[1]: Starting User Login Management...
client # [348140.978497] client systemd[1]: Starting Permit User Sessions...
client # [348141.016344] client systemd[1]: Finished Permit User Sessions.
client # [348141.017569] client systemd[1]: Started Console Getty.
client # [348141.017622] client systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0
client # [348141.017647] client systemd[1]: Reached target Login Prompts.
server # [348140.788592] server systemd-resolved[140]: Positive Trust Anchors:
server # [348140.788607] server systemd-resolved[140]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
server # [348140.788611] server systemd-resolved[140]: . IN DS 38696 8 2 683d2d0acb8c9b712a1948b27f741219298d0a450d612c483af444a4c0fb2b16
server # [348140.788645] server systemd-resolved[140]: Negative trust anchors: home.arpa 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 170.0.0.192.in-addr.arpa 171.0.0.192.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa ipv4only.arpa resolver.arpa corp home internal intranet lan local private test
server # [348140.789390] server systemd-resolved[140]: Using system hostname 'server'.
server # [348140.790861] server systemd[1]: Started Network Name Resolution.
server # [348140.790957] server systemd[1]: Reached target Network.
server # [348140.791039] server systemd[1]: Reached target System Initialization.
server # [348140.791126] server systemd[1]: Started Watch for zone file changes.
server # [348140.791159] server systemd[1]: Discard unused filesystem blocks once a week skipped, unmet condition check ConditionVirtualization=!container
server # [348140.791186] server systemd[1]: Started Daily Cleanup of Temporary Directories.
server # [348140.791209] server systemd[1]: Reached target Path Units.
server # [348140.791245] server systemd[1]: Reached target Timer Units.
server # [348140.791377] server systemd[1]: Listening on D-Bus System Message Bus Socket.
server # [348140.791499] server systemd[1]: Listening on Nix Daemon Socket.
server # [348140.791614] server systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.
server # [348140.791637] server systemd[1]: Reached target Socket Units.
server # [348140.791684] server systemd[1]: Reached target Basic System.
server # [348140.793395] server systemd[1]: Starting data mesher daemon...
server # [348140.794282] server systemd[1]: Starting Import lastlog data into lastlog2 database...
server # [348140.795378] server systemd[1]: Starting Name Service Cache Daemon (nsncd)...
server # [348140.797105] server systemd[1]: Starting D-Bus System Message Bus...
server # [348140.815665] server systemd[1]: Finished Import lastlog data into lastlog2 database.
server # [348140.983442] server systemd[1]: Started Name Service Cache Daemon (nsncd).
server # [348140.983571] server systemd[1]: Reached target User and Group Name Lookups.
server # [348140.983649] server nsncd[225]: Oct 20 10:05:45.955 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"
server # [348141.008806] server systemd[1]: Starting User Login Management...
server # [348141.010256] server systemd[1]: Starting Permit User Sessions...
server # [348141.023132] server systemd[1]: Finished Permit User Sessions.
server # [348141.024779] server systemd[1]: Started Console Getty.
server # [348141.024847] server systemd[1]: Getty on tty1 skipped, unmet condition check ConditionPathExists=/dev/tty0
server # [348141.024883] server systemd[1]: Reached target Login Prompts.
server # [348141.147070] server dbus-broker-launch[226]: Looking up NSS user entry for 'systemd-timesync'...
server # [348141.148290] server dbus-broker-launch[226]: NSS returned no entry for 'systemd-timesync'
server # [348141.148290] server dbus-broker-launch[226]: Invalid user-name in /nix/store/magy4l0prhx3hshr0pd5ipj48dd0nql3-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"
server # [348141.149124] server systemd[1]: Started D-Bus System Message Bus.
server # [348141.157723] server dbus-broker-launch[226]: Ready
client # [348141.126010] client dbus-broker-launch[217]: Looking up NSS user entry for 'systemd-timesync'...
client # [348141.127358] client dbus-broker-launch[217]: NSS returned no entry for 'systemd-timesync'
client # [348141.127358] client dbus-broker-launch[217]: Invalid user-name in /nix/store/fg6hynvdpnsy8hqa5zxbswggn0y8zf3g-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"
client # [348141.127901] client systemd[1]: Started D-Bus System Message Bus.
client # [348141.137554] client dbus-broker-launch[217]: Ready
server # [348141.589165] server data-mesher[223]: time=2026-10-20T10:05:46.560Z level=DEBUG msg="http config" listen_addresses="[[::1]:7331 127.0.0.1:7331]" port=7331 interfaces=[lo]
server # [348141.590130] server data-mesher[223]: time=2026-10-20T10:05:46.561Z level=DEBUG msg="cluster config" port=7946 interfaces=[] bootstrap_peers="[{12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P: [/dns/client.test/tcp/7946]} {12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA: [/dns/server.test/tcp/7946]}]" push_pull_interval=5s peer_id=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA
server # [348141.590130] server data-mesher[223]: time=2026-10-20T10:05:46.562Z level=INFO msg="config loaded" config_file=/etc/data-mesher/dm.toml
server # [348141.594760] server data-mesher[223]: time=2026-10-20T10:05:46.566Z level=INFO msg="checking file integrity"
server # [348141.594886] server data-mesher[223]: time=2026-10-20T10:05:46.566Z level=INFO msg="file integrity check complete"
server # [348141.598861] server data-mesher[223]: time=2026-10-20T10:05:46.570Z level=INFO msg="libp2p host created" peer_id=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA addresses="[/ip4/127.0.0.1/tcp/7946 /ip4/192.168.1.2/tcp/7946 /ip6/::1/tcp/7946 /ip6/2001:db8:1::2/tcp/7946]"
server # [348141.598939] server data-mesher[223]: time=2026-10-20T10:05:46.570Z level=INFO msg="registered HTTP route" method=GET path=/files
server # [348141.598939] server data-mesher[223]: time=2026-10-20T10:05:46.570Z level=INFO msg="registered HTTP route" method=PUT path=/files/:name
server # [348141.598939] server data-mesher[223]: time=2026-10-20T10:05:46.570Z level=INFO msg="registered HTTP route" method=DELETE path=/files/:name
server # [348141.598939] server data-mesher[223]: time=2026-10-20T10:05:46.570Z level=INFO msg="starting server"
server # [348141.599125] server data-mesher[223]: time=2026-10-20T10:05:46.570Z level=INFO msg="waiting for DHT to populate" delay=10s
server # [348141.599559] server data-mesher[223]: time=2026-10-20T10:05:46.571Z level=INFO msg="HTTP server listening" address=[::1]:7331
server # [348141.599618] server data-mesher[223]: time=2026-10-20T10:05:46.571Z level=INFO msg="HTTP server listening" address=127.0.0.1:7331
server # [348141.604911] server data-mesher[223]: time=2026-10-20T10:05:46.576Z level=INFO msg="peer connected" peer_id=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P remote_addr=/ip4/192.168.1.1/tcp/7946
server # [348141.633226] server data-mesher[223]: time=2026-10-20T10:05:46.605Z level=INFO msg="peer connected" peer_id=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P remote_addr=/ip4/192.168.1.1/tcp/52610
client # [348141.589168] client data-mesher[214]: time=2026-10-20T10:05:46.560Z level=DEBUG msg="http config" listen_addresses="[[::1]:7331 127.0.0.1:7331]" port=7331 interfaces=[lo]
client # [348141.590152] client data-mesher[214]: time=2026-10-20T10:05:46.561Z level=DEBUG msg="cluster config" port=7946 interfaces=[] bootstrap_peers="[{12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P: [/dns/client.test/tcp/7946]} {12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA: [/dns/server.test/tcp/7946]}]" push_pull_interval=5s peer_id=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P
client # [348141.590152] client data-mesher[214]: time=2026-10-20T10:05:46.562Z level=INFO msg="config loaded" config_file=/etc/data-mesher/dm.toml
client # [348141.594727] client data-mesher[214]: time=2026-10-20T10:05:46.566Z level=INFO msg="checking file integrity"
client # [348141.594887] client data-mesher[214]: time=2026-10-20T10:05:46.566Z level=INFO msg="file integrity check complete"
client # [348141.598855] client data-mesher[214]: time=2026-10-20T10:05:46.570Z level=INFO msg="libp2p host created" peer_id=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P addresses="[/ip4/127.0.0.1/tcp/7946 /ip4/192.168.1.1/tcp/7946 /ip6/::1/tcp/7946 /ip6/2001:db8:1::1/tcp/7946]"
client # [348141.598912] client data-mesher[214]: time=2026-10-20T10:05:46.570Z level=INFO msg="registered HTTP route" method=GET path=/files
client # [348141.598912] client data-mesher[214]: time=2026-10-20T10:05:46.570Z level=INFO msg="registered HTTP route" method=PUT path=/files/:name
client # [348141.598912] client data-mesher[214]: time=2026-10-20T10:05:46.570Z level=INFO msg="registered HTTP route" method=DELETE path=/files/:name
client # [348141.598912] client data-mesher[214]: time=2026-10-20T10:05:46.570Z level=INFO msg="starting server"
client # [348141.599036] client data-mesher[214]: time=2026-10-20T10:05:46.570Z level=INFO msg="waiting for DHT to populate" delay=10s
client # [348141.599134] client data-mesher[214]: time=2026-10-20T10:05:46.570Z level=INFO msg="HTTP server listening" address=[::1]:7331
client # [348141.599189] client data-mesher[214]: time=2026-10-20T10:05:46.571Z level=INFO msg="HTTP server listening" address=127.0.0.1:7331
client # [348141.606397] client data-mesher[214]: time=2026-10-20T10:05:46.578Z level=INFO msg="peer connected" peer_id=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA remote_addr=/ip4/192.168.1.2/tcp/7946
client # [348141.632233] client systemd-networkd[209]: eth1: Gained IPv6LL
client # [348141.632592] client data-mesher[214]: time=2026-10-20T10:05:46.604Z level=INFO msg="peer connected" peer_id=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA remote_addr=/ip4/192.168.1.2/tcp/7946
client # [348141.837938] client systemd-logind[234]: New seat seat0.
client # [348141.838223] client systemd[1]: Started User Login Management.
server # [348141.847384] server systemd-logind[243]: New seat seat0.
server # [348141.847730] server systemd[1]: Started User Login Management.
server # [348141.865359] server systemd[1]: Starting linger-users.service...
server # [348141.881047] server systemd[1]: linger-users.service: Failed to read pids.max for control group /system.slice/linger-users.service, ignoring: No data available
server # [348141.881152] server systemd[1]: linger-users.service: Failed to read pids.max for control group /system.slice/linger-users.service, ignoring: No data available
server # [348141.881219] server systemd[1]: linger-users.service: Deactivated successfully.
server # [348141.881474] server systemd[1]: Finished linger-users.service.
server # [348141.980123] server systemd-networkd[218]: eth1: Gained IPv6LL
client # [348141.865112] client systemd[1]: Starting linger-users.service...
client # [348141.887712] client systemd[1]: linger-users.service: Failed to read pids.max for control group /system.slice/linger-users.service, ignoring: No data available
client # [348141.887757] client systemd[1]: linger-users.service: Failed to read pids.max for control group /system.slice/linger-users.service, ignoring: No data available
client # [348141.887789] client systemd[1]: linger-users.service: Deactivated successfully.
client # [348141.887908] client systemd[1]: Finished linger-users.service.
server: still waiting for container 'server' to reach ready state...
server # [348151.599914] server data-mesher[223]: time=2026-10-20T10:05:56.571Z level=INFO msg="performing state exchange with peers on join" count=1
server # [348151.599914] server data-mesher[223]: time=2026-10-20T10:05:56.571Z level=DEBUG msg="initiating state exchange" peer=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P timeout=5s
server # [348151.600733] server data-mesher[223]: time=2026-10-20T10:05:56.572Z level=INFO msg="received state sync from peer" peer=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P
server # [348151.600733] server data-mesher[223]: time=2026-10-20T10:05:56.572Z level=INFO msg="merging remote state" peer=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P
server # [348151.600979] server data-mesher[223]: time=2026-10-20T10:05:56.572Z level=INFO msg="merging remote state" peer=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P
server # [348151.600979] server data-mesher[223]: time=2026-10-20T10:05:56.572Z level=INFO msg="state exchange complete" peer=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P timeout=5s
server # [348151.601091] server data-mesher[223]: time=2026-10-20T10:05:56.572Z level=INFO msg="server started"
server # [348151.601229] server data-mesher[223]: time=2026-10-20T10:05:56.573Z level=INFO msg="starting expired-file sweeper" interval=1m0s
server # [348151.601325] server systemd[1]: Started data mesher daemon.
server # [348151.603919] server systemd[1]: Starting Unbound recursive Domain Name Server...
client # [348151.599442] client data-mesher[214]: time=2026-10-20T10:05:56.571Z level=INFO msg="performing state exchange with peers on join" count=1
client # [348151.599442] client data-mesher[214]: time=2026-10-20T10:05:56.571Z level=DEBUG msg="initiating state exchange" peer=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA timeout=5s
client # [348151.600554] client data-mesher[214]: time=2026-10-20T10:05:56.572Z level=INFO msg="merging remote state" peer=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA
client # [348151.600554] client data-mesher[214]: time=2026-10-20T10:05:56.572Z level=INFO msg="state exchange complete" peer=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA timeout=5s
client # [348151.600708] client data-mesher[214]: time=2026-10-20T10:05:56.572Z level=INFO msg="received state sync from peer" peer=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA
client # [348151.600708] client data-mesher[214]: time=2026-10-20T10:05:56.572Z level=INFO msg="merging remote state" peer=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA
client # [348151.600829] client data-mesher[214]: time=2026-10-20T10:05:56.572Z level=INFO msg="server started"
client # [348151.600900] client data-mesher[214]: time=2026-10-20T10:05:56.572Z level=INFO msg="starting expired-file sweeper" interval=1m0s
client # [348151.601060] client systemd[1]: Started data mesher daemon.
client # [348151.603639] client systemd[1]: Starting Unbound recursive Domain Name Server...
client # [348152.505093] client unbound-pre-start[278]: Root anchor updated!
client # [348152.520487] client unbound-pre-start[282]: setup in directory /var/lib/unbound
server # [348152.500068] server unbound-pre-start[286]: Root anchor updated!
server # [348152.515971] server unbound-pre-start[290]: setup in directory /var/lib/unbound
server # [348152.914779] server unbound-pre-start[299]: Certificate request self-signature ok
server # [348152.914779] server unbound-pre-start[299]: subject=CN=unbound-control
server # [348152.935426] server unbound-pre-start[290]: removing artifacts
server # [348152.937397] server unbound-pre-start[290]: Setup success. Certificates created. Enable in unbound.conf file to use
client # [348153.601373] client unbound-pre-start[291]: Certificate request self-signature ok
client # [348153.601373] client unbound-pre-start[291]: subject=CN=unbound-control
client # [348153.621422] client unbound-pre-start[282]: removing artifacts
client # [348153.623213] client unbound-pre-start[282]: Setup success. Certificates created. Enable in unbound.conf file to use
server # [348153.778686] server unbound[303]: [303:0] notice: init module 0: validator
server # [348153.778762] server unbound[303]: [303:0] notice: init module 1: iterator
server # [348153.782929] server unbound[303]: [303:0] info: start of service (unbound 1.26.1).
server # [348153.783086] server systemd[1]: Started Unbound recursive Domain Name Server.
server # [348153.783605] server systemd[1]: Reached target Multi-User System.
server # [348153.783881] server systemd[1]: Reached target Host and Network Name Lookups.
server # [348153.785744] server systemd[1]: Starting Reload unbound zone configuration...
server # [348153.833697] server unbound[303]: [303:0] info: service stopped (unbound 1.26.1).
server # [348153.834033] server unbound-control[307]: ok
server # [348153.834075] server unbound[303]: [303:0] info: server stats for thread 0: 1 queries, 0 answers from cache, 1 recursions, 0 prefetch, 0 rejected by ip ratelimiting
server # [348153.834079] server unbound[303]: [303:0] info: server stats for thread 0: requestlist max 1 avg 1 exceeded 0 jostled 0
server # [348153.835112] server unbound[303]: [303:0] notice: Restart of unbound 1.26.1.
server # [348153.835833] server unbound[303]: [303:0] notice: init module 0: validator
server # [348153.835888] server unbound[303]: [303:0] notice: init module 1: iterator
server # [348153.836428] server systemd[1]: unbound-reload-zones.service: Failed to read pids.max for control group /system.slice/unbound-reload-zones.service, ignoring: No data available
server # [348153.836478] server systemd[1]: unbound-reload-zones.service: Failed to read pids.max for control group /system.slice/unbound-reload-zones.service, ignoring: No data available
server # [348153.836506] server systemd[1]: unbound-reload-zones.service: Deactivated successfully.
server # [348153.836731] server systemd[1]: Finished Reload unbound zone configuration.
server # [348153.839091] server systemd[1]: Startup finished in 14.747s.
server # [348153.839798] server unbound[303]: [303:0] info: start of service (unbound 1.26.1).
server: (finished: waiting for unit unbound.service, in 15.67 seconds)
client: waiting for unit unbound.service
client: (finished: waiting for unit unbound.service, in 0.33 seconds)
server: waiting for unit data-mesher.service
server: (finished: waiting for unit data-mesher.service, in 0.02 seconds)
server: waiting for success: dig +short @127.0.0.1 -p 5353 server.test A | grep 10.0.0.1
server: (finished: waiting for success: dig +short @127.0.0.1 -p 5353 server.test A | grep 10.0.0.1, in 0.03 seconds)
server: must succeed: data-mesher file update --network-id /nix/store/7762dj40jikvbdy0n7ibf5rysq1cgdip-shared-data-mesher-network_network.pub /nix/store/4g9kvmaqs34gisvxgf61p42p665lx8dx-shared-dm-dns_zone.conf --url http://localhost:7331 --key /run/secrets/shared/dm-dns-signing-key/signing.key --name dns/cnames
server: (finished: must succeed: data-mesher file update --network-id /nix/store/7762dj40jikvbdy0n7ibf5rysq1cgdip-shared-data-mesher-network_network.pub /nix/store/4g9kvmaqs34gisvxgf61p42p665lx8dx-shared-dm-dns_zone.conf --url http://localhost:7331 --key /run/secrets/shared/dm-dns-signing-key/signing.key --name dns/cnames, in 0.03 seconds)
??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.
    File "/nix/store/3cbsvnfi3n2bcbv8am285hmkrj1a9kcg-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39
server: waiting for success: dig +short @127.0.0.1 -p 5353 myapp.test CNAME | grep server.test
??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.
    File "/nix/store/3cbsvnfi3n2bcbv8am285hmkrj1a9kcg-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 39
client # [348154.455221] client unbound[295]: [295:0] notice: init module 0: validator
client # [348154.455304] client unbound[295]: [295:0] notice: init module 1: iterator
client # [348154.459496] client unbound[295]: [295:0] info: start of service (unbound 1.26.1).
client # [348154.459651] client systemd[1]: Started Unbound recursive Domain Name Server.
client # [348154.460186] client systemd[1]: Reached target Multi-User System.
client # [348154.460477] client systemd[1]: Reached target Host and Network Name Lookups.
client # [348154.462330] client systemd[1]: Starting Reload unbound zone configuration...
client # [348154.505210] client unbound[295]: [295:0] info: service stopped (unbound 1.26.1).
client # [348154.505553] client unbound-control[299]: ok
client # [348154.505590] client unbound[295]: [295:0] info: server stats for thread 0: 1 queries, 0 answers from cache, 1 recursions, 0 prefetch, 0 rejected by ip ratelimiting
client # [348154.505595] client unbound[295]: [295:0] info: server stats for thread 0: requestlist max 1 avg 1 exceeded 0 jostled 0
client # [348154.506580] client unbound[295]: [295:0] notice: Restart of unbound 1.26.1.
client # [348154.507111] client systemd[1]: unbound-reload-zones.service: Failed to read pids.max for control group /system.slice/unbound-reload-zones.service, ignoring: No data available
client # [348154.507209] client systemd[1]: unbound-reload-zones.service: Failed to read pids.max for control group /system.slice/unbound-reload-zones.service, ignoring: No data available
client # [348154.507262] client systemd[1]: unbound-reload-zones.service: Deactivated successfully.
client # [348154.507331] client unbound[295]: [295:0] notice: init module 0: validator
client # [348154.507389] client unbound[295]: [295:0] notice: init module 1: iterator
client # [348154.507585] client systemd[1]: Finished Reload unbound zone configuration.
client # [348154.508156] client systemd[1]: Startup finished in 15.422s.
client # [348154.511443] client unbound[295]: [295:0] info: start of service (unbound 1.26.1).
server # [348154.617918] server data-mesher[223]: time=2026-10-20T10:05:59.589Z level=INFO msg=http_request uri=/files/dns/cnames status=204
server # [348154.620703] server systemd[1]: Starting Reload unbound zone configuration...
server # [348154.673922] server unbound-control[342]: ok
server # [348154.674021] server unbound[303]: [303:0] info: service stopped (unbound 1.26.1).
server # [348154.674501] server unbound[303]: [303:0] info: server stats for thread 0: 5 queries, 2 answers from cache, 3 recursions, 0 prefetch, 0 rejected by ip ratelimiting
server # [348154.674509] server unbound[303]: [303:0] info: server stats for thread 0: requestlist max 2 avg 1.33333 exceeded 0 jostled 0
server # [348154.675680] server unbound[303]: [303:0] notice: Restart of unbound 1.26.1.
server # [348154.676269] server systemd[1]: unbound-reload-zones.service: Failed to read pids.max for control group /system.slice/unbound-reload-zones.service, ignoring: No data available
server # [348154.676324] server systemd[1]: unbound-reload-zones.service: Failed to read pids.max for control group /system.slice/unbound-reload-zones.service, ignoring: No data available
server # [348154.676355] server systemd[1]: unbound-reload-zones.service: Deactivated successfully.
server # [348154.676538] server systemd[1]: Finished Reload unbound zone configuration.
server # [348154.676770] server unbound[303]: [303:0] notice: init module 0: validator
server # [348154.676840] server unbound[303]: [303:0] notice: init module 1: iterator
server # [348154.681819] server unbound[303]: [303:0] info: start of service (unbound 1.26.1).
server: (finished: waiting for success: dig +short @127.0.0.1 -p 5353 myapp.test CNAME | grep server.test, in 1.06 seconds)
(finished: run the VM test script, in 17.14 seconds)
server # [348156.601768] server data-mesher[223]: time=2026-10-20T10:06:01.573Z level=DEBUG msg="attempting push/pull" peer_count=1
server # [348156.601768] server data-mesher[223]: time=2026-10-20T10:06:01.573Z level=DEBUG msg="initiating state exchange" peer=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P timeout=5s
server # [348156.601768] server data-mesher[223]: time=2026-10-20T10:06:01.573Z level=INFO msg="received state sync from peer" peer=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P
server # [348156.601768] server data-mesher[223]: time=2026-10-20T10:06:01.573Z level=INFO msg="merging remote state" peer=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P
server # [348156.602530] server data-mesher[223]: time=2026-10-20T10:06:01.574Z level=INFO msg="merging remote state" peer=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P
server # [348156.602530] server data-mesher[223]: time=2026-10-20T10:06:01.574Z level=INFO msg="state exchange complete" peer=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P timeout=5s
server # [348156.602530] server data-mesher[223]: time=2026-10-20T10:06:01.574Z level=DEBUG msg="push/pull successful" interval=5s
server # [348156.602820] server data-mesher[223]: time=2026-10-20T10:06:01.574Z level=INFO msg="received file request" peer=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P network="1jXWvclSk1vLoLM5SbzwG8LoKt8JHPMuQXFTo7uZufI=" name=dns/cnames
server # [348156.604927] server data-mesher[223]: time=2026-10-20T10:06:01.576Z level=INFO msg="file transfer complete" peer=12D3KooWPhLj6r1eiXnRqo7sjW8WvWEyAgyee8nsbc1qvxt2Uc6P network="1jXWvclSk1vLoLM5SbzwG8LoKt8JHPMuQXFTo7uZufI=" name=dns/cnames
test script finished in 18.46s
cleanup
kill NspawnMachine (pid 52)
client # [348156.601187] client data-mesher[214]: time=2026-10-20T10:06:01.573Z level=DEBUG msg="attempting push/pull" peer_count=1
client # [348156.601567] client data-mesher[214]: time=2026-10-20T10:06:01.573Z level=DEBUG msg="initiating state exchange" peer=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA timeout=5s
client # [348156.602106] client data-mesher[214]: time=2026-10-20T10:06:01.573Z level=INFO msg="merging remote state" peer=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA
client # [348156.602137] client data-mesher[214]: time=2026-10-20T10:06:01.573Z level=DEBUG msg="new file detected" peer=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA name=dns/cnames name=dns/cnames
client # [348156.602137] client data-mesher[214]: time=2026-10-20T10:06:01.574Z level=INFO msg="received state sync from peer" peer=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA
client # [348156.602180] client data-mesher[214]: time=2026-10-20T10:06:01.574Z level=INFO msg="state exchange complete" peer=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA timeout=5s
client # [348156.602180] client data-mesher[214]: time=2026-10-20T10:06:01.574Z level=DEBUG msg="push/pull successful" interval=5s
client # [348156.602180] client data-mesher[214]: time=2026-10-20T10:06:01.574Z level=INFO msg="merging remote state" peer=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA
client # [348156.602247] client data-mesher[214]: time=2026-10-20T10:06:01.574Z level=DEBUG msg="new file detected" peer=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA name=dns/cnames name=dns/cnames
client # [348156.602274] client data-mesher[214]: time=2026-10-20T10:06:01.574Z level=INFO msg="scheduling file download" name=dns/cnames
client # [348156.602347] client data-mesher[214]: time=2026-10-20T10:06:01.574Z level=INFO msg="downloading file" name=dns/cnames signed_at="2026-10-20 10:05:59.586 +0000 UTC" signed_by="p4JbSx0rDRo/IEa/0cb5ctpFGEYPY0yRH3c8GzY0/l4=" peer=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA
client # [348156.609050] client systemd[1]: Starting Reload unbound zone configuration...
client # [348156.611143] client data-mesher[214]: time=2026-10-20T10:06:01.582Z level=INFO msg="download complete" name=dns/cnames signed_at="2026-10-20 10:05:59.586 +0000 UTC" signed_by="p4JbSx0rDRo/IEa/0cb5ctpFGEYPY0yRH3c8GzY0/l4=" peer=12D3KooWGc4HvqYPC6cZzk9UiYo9u9qtmGodpKwdr41upEd6dPCA written=true elapsed=8.837801ms
client # [348156.667495] client unbound-control[304]: ok
client # [348156.667562] client unbound[295]: [295:0] info: service stopped (unbound 1.26.1).
client # [348156.668228] client unbound[295]: [295:0] info: server stats for thread 0: 5 queries, 0 answers from cache, 5 recursions, 0 prefetch, 0 rejected by ip ratelimiting
client # [348156.668241] client unbound[295]: [295:0] info: server stats for thread 0: requestlist max 2 avg 1.6 exceeded 0 jostled 0
client # [348156.668741] client systemd[1]: unbound-reload-zones.service: Failed to read pids.max for control group /system.slice/unbound-reload-zones.service, ignoring: No data available
client # [348156.668776] client systemd[1]: unbound-reload-zones.service: Failed to read pids.max for control group /system.slice/unbound-reload-zones.service, ignoring: No data available
client # [348156.668796] client systemd[1]: unbound-reload-zones.service: Deactivated successfully.
client # [348156.669034] client systemd[1]: Finished Reload unbound zone configuration.
client # [348156.670041] client unbound[295]: [295:0] notice: Restart of unbound 1.26.1.
client # [348156.671738] client unbound[295]: [295:0] notice: init module 0: validator
client # [348156.671845] client unbound[295]: [295:0] notice: init module 1: iterator
client # [348156.681442] client unbound[295]: [295:0] info: start of service (unbound 1.26.1).
client # [348156.998146] client systemd-networkd[209]: eth1: Link DOWN
client # [348156.998174] client systemd-networkd[209]: eth1: Lost carrier
kill NspawnMachine (pid 53)
Container client terminated by signal KILL.
(finished: cleanup, in 0.28 seconds)
Container server terminated by signal KILL.