Skip to content

feat: IPv6 header build/parse (roadmap task 1) - #49

Merged
gspivey merged 2 commits into
developmentfrom
agent/ipv6-header-build-parse
Apr 25, 2026
Merged

gspivey merged 2 commits into
developmentfrom
agent/ipv6-header-build-parse

Conversation

@gspivey

@gspivey gspivey commented Apr 25, 2026

Copy link
Copy Markdown
Owner

Roadmap Item

IPv6 task 1: header build/parse — the first step toward full IPv6 support in dpdk-stdlib-rust.

What This Provides

  • Frame builders: build_ipv6_udp_frame() and build_ipv6_udp_frame_into() for constructing IPv6/UDP Ethernet frames with proper headers
  • Parsers: parse_ipv6_udp_frame() and zero-copy Ipv6UdpFrameRef for parsing inbound IPv6/UDP frames, including VLAN-tagged frames
  • Checksums: UDP-over-IPv6 pseudo-header checksum (RFC 2460 §8.1) with validation — mandatory for IPv6 (no zero-means-disabled like IPv4)
  • Extension header walk: walk_extension_headers() traverses Hop-by-Hop, Routing, Fragment, Destination Options, and AH headers to locate the upper-layer protocol
  • Constants: IPV6_HEADER_LEN, ETH_IPV6_UDP_HEADER_LEN, MAX_IPV6_UDP_PAYLOAD, EtherType, Next Header values

Test Coverage

  • 34 new unit tests covering build/parse roundtrips, wire format verification, checksum validation, extension header walking, error cases, VLAN support, and a build/parse performance cycle
  • 422 total tests passing across all crates (51 dpdk + 3 dpdk-sys + 8 dpdk-tokio + 353 dpdk-udp + 2 signal_shutdown + 5 synthetic_bench)

Notes

This resumes work from a prior agent-router session that was blocked on credentials (GITHUB_TOKEN was not in the daemon environment). The implementation and tests were complete and passing locally; this PR ships that work now that auth is fixed.

Add dpdk-udp/src/ipv6.rs implementing IPv6 roadmap task 1:

Build functions:
- build_udp6_frame() — allocating frame builder (Vec<u8>)
- build_udp6_frame_into() — zero-alloc hot-path builder

Parse functions:
- parse_udp6_packet() — allocating parser (owns payload)
- parse_udp6_packet_ref() — zero-copy parser (borrows payload)

Checksum:
- udp6_checksum() — mandatory IPv6 UDP checksum (RFC 8200 §8.1)
- udp6_pseudo_header_checksum() — for TX hardware offload
- verify_udp6_checksum() — RX validation (rejects zero checksum)

Extension header walking:
- walk_extension_headers() — chain walk for Hop-by-Hop, Routing,
  Fragment, and Destination Options headers to locate L4 payload

34 tests covering: roundtrip build+parse, extension header chains,
checksum validation/corruption, wire format, VLAN-tagged frames,
special addresses, edge cases, and a synthetic perf benchmark.

All 422 tests pass (388 existing + 34 new).
@github-actions

Copy link
Copy Markdown

Synthetic Performance Results — Graviton (run)

Commit: 78dac40d

✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)

Synthetic UDP Performance Results

Measures framework overhead: sync dpdk_udp::UdpSocket vs async (std::sync::Mutex + try_recv_from).

Test Payload Sync PPS Async PPS Ratio (sync/async) Async ns/op
TX send_to 64B 11.6M 11.1M 1.0x 90
RX recv_from 64B 3.4M 4.4M 0.8x 228
TX send_to 1400B 1.6M 1.6M 1.0x 612
RX recv_from 1400B 981.3K 1.1M 0.9x 946

Avg sync/async ratio: 0.9x, worst: 1.0x

Good: Async wrapper is within 1.0x of sync — minimal framework overhead.

@github-actions

Copy link
Copy Markdown

Synthetic Performance Results (run)

Commit: 78dac40d

✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)

Synthetic UDP Performance Results

Measures framework overhead: sync dpdk_udp::UdpSocket vs async (std::sync::Mutex + try_recv_from).

Test Payload Sync PPS Async PPS Ratio (sync/async) Async ns/op
TX send_to 64B 12.1M 11.4M 1.1x 87
RX recv_from 64B 3.7M 4.9M 0.8x 202
TX send_to 1400B 1.8M 1.8M 1.0x 553
RX recv_from 1400B 1.2M 1.3M 0.9x 791

Avg sync/async ratio: 0.9x, worst: 1.1x

Good: Async wrapper is within 1.1x of sync — minimal framework overhead.

@github-actions

Copy link
Copy Markdown

[CI] Stage: Deploy

Infrastructure ready.

  • Sender: i-00c0e4441dabe667d (DPDK ENI: 10.0.1.206)
  • Receiver: i-049934f19df763cf0 (DPDK ENI: 10.0.1.246)
  • Both instances SSM-ready.

@github-actions

Copy link
Copy Markdown

[CI] Stage: Deploy

Infrastructure ready.

  • Sender: i-039ef5a00d19ff0ac (DPDK ENI: 10.0.1.200)
  • Receiver: i-07a348ac0a0b105a2 (DPDK ENI: 10.0.1.83)
  • Both instances SSM-ready.

@github-actions

Copy link
Copy Markdown

[CI] Stage: Summary

All tests PASSED.

ARP seeding: kernel /proc/net/arp (automatic)

  • tier1-dpdk-echo: 6 tests, 0 failures
  • tier2-kernel-interop: 4 tests, 0 failures
  • tier3-iperf-interop: 1 tests, 0 failures
  • tier3-iperf-interop: 1 tests, 0 failures

@github-actions

Copy link
Copy Markdown

✅ Integration Tests Passed — Graviton (run)

Branch: 49/merge | Commit: 78dac40d

Test Results

  • tier1-dpdk-echo.xml: 6 tests, 0 failures
  • tier2-kernel-interop.xml: 4 tests, 0 failures
  • tier3-iperf-sends.xml: 1 tests, 0 failures
  • tier3-our-app-sends.xml: 1 tests, 0 failures

Application Logs (last 20 lines)

receiver-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.246:9000 (MAC: 02:2a:4a:d3:cf:5d)
echo listening on 10.0.1.246:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.206:9000 (MAC: 02:6d:75:2b:38:df)
echo listening on 10.0.1.206:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-test-client.log

EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
TELEMETRY: No legacy callbacks, legacy socket not created
DPDK bind failed (Port init failed: Invalid port ID: 0), falling back to tokio
Backend: tokio
Sending packets...
Sent 12 bytes: 'arp-probe #1'
Received 12 bytes from 10.0.1.246:9000: 'arp-probe #1'
Test complete
[2026-04-25T23:35:35Z] INFO: ARP resolution succeeded (got response from peer)
[2026-04-25T23:35:35Z] INFO: Test: udp_send_receive
[2026-04-25T23:35:36Z] INFO: UDP send/receive succeeded
[2026-04-25T23:35:36Z] INFO: Test: echo_roundtrip
[2026-04-25T23:35:37Z] INFO: Echo roundtrip succeeded: 5/5 responses received
[2026-04-25T23:35:37Z] INFO: Test: payload_integrity
[2026-04-25T23:35:37Z] INFO: Response received, checking payload match...
[2026-04-25T23:35:37Z] INFO: Payload integrity verified (found in response)
[2026-04-25T23:35:37Z] INFO: JUnit XML written to: /tmp/test-results/tier2-kernel-interop.xml
[2026-04-25T23:35:37Z] INFO: Tier 2 sender tests complete. Results: /tmp/test-results/tier2-kernel-interop.xml

@github-actions

Copy link
Copy Markdown

[CI] Stage: Summary

All tests PASSED.

ARP seeding: kernel /proc/net/arp (automatic)

  • tier1-dpdk-echo: 6 tests, 0 failures
  • tier2-kernel-interop: 4 tests, 0 failures
  • tier3-iperf-interop: 1 tests, 0 failures
  • tier3-iperf-interop: 1 tests, 0 failures

@github-actions

Copy link
Copy Markdown

✅ Integration Tests Passed (Run 24943086329)

Branch: 49/merge | Commit: 78dac40d

Test Results

  • tier1-dpdk-echo.xml: 6 tests, 0 failures, skipped
  • tier2-kernel-interop.xml: 4 tests, 0 failures, skipped
  • tier3-iperf-sends.xml: 1 tests, 0 failures, skipped
  • tier3-our-app-sends.xml: 1 tests, 0 failures, skipped

Application Logs (last 20 lines)

receiver-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.83:9000 (MAC: 02:34:43:89:d8:2d)
echo listening on 10.0.1.83:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.200:9000 (MAC: 02:ff:aa:49:4f:df)
echo listening on 10.0.1.200:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-test-client.log

EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
TELEMETRY: No legacy callbacks, legacy socket not created
DPDK bind failed (Port init failed: Invalid port ID: 0), falling back to tokio
Backend: tokio
Sending packets...
Sent 12 bytes: 'arp-probe #1'
Received 12 bytes from 10.0.1.83:9000: 'arp-probe #1'
Test complete
[2026-04-25T23:38:34Z] INFO: ARP resolution succeeded (got response from peer)
[2026-04-25T23:38:34Z] INFO: Test: udp_send_receive
[2026-04-25T23:38:35Z] INFO: UDP send/receive succeeded
[2026-04-25T23:38:35Z] INFO: Test: echo_roundtrip
[2026-04-25T23:38:36Z] INFO: Echo roundtrip succeeded: 5/5 responses received
[2026-04-25T23:38:36Z] INFO: Test: payload_integrity
[2026-04-25T23:38:36Z] INFO: Response received, checking payload match...
[2026-04-25T23:38:36Z] INFO: Payload integrity verified (found in response)
[2026-04-25T23:38:36Z] INFO: JUnit XML written to: /tmp/test-results/tier2-kernel-interop.xml
[2026-04-25T23:38:36Z] INFO: Tier 2 sender tests complete. Results: /tmp/test-results/tier2-kernel-interop.xml

receiver-test-client-iperf.log

[2026-04-25T23:43:39Z] INFO: iperf-sends: sent 10 packets, received 10 responses
[2026-04-25T23:43:39Z] INFO: iperf-sends: PASS (sent >= 5 packets)
[2026-04-25T23:43:39Z] INFO: JUnit XML written to: /tmp/test-results/tier3-iperf-sends.xml
[2026-04-25T23:43:39Z] INFO: iperf-sends test complete

sender-test-client-iperf.log

Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #3'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #4'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #4'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #5'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #5'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #6'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #6'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #7'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #7'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #8'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #8'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #9'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #9'
Sent 31 bytes: 'dpdk-to-kernel-test-payload #10'
Received 31 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #10'
Test complete
[2026-04-25T23:42:38Z] INFO: our-app-sends: sent 10 packets, received 10 responses
[2026-04-25T23:42:38Z] INFO: our-app-sends: PASS (sent >= 5 packets)
[2026-04-25T23:42:38Z] INFO: JUnit XML written to: /tmp/test-results/tier3-our-app-sends.xml
[2026-04-25T23:42:38Z] INFO: our-app-sends test complete
Full Application Logs (last 200 lines each)

receiver-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.83:9000 (MAC: 02:34:43:89:d8:2d)
echo listening on 10.0.1.83:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.200:9000 (MAC: 02:ff:aa:49:4f:df)
echo listening on 10.0.1.200:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-test-client.log

[2026-04-25T23:31:29Z] INFO: Test: arp_resolution
UDP Test Client
Target: 10.0.1.83:9000
Bind address: 10.0.1.200:0
Message: 'arp-probe'
Count: 1
EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.200:32768 (MAC: 02:ff:aa:49:4f:df)
Backend: dpdk
Sending packets...
Sent 12 bytes: 'arp-probe #1'
Received 12 bytes from 10.0.1.83:9000: 'arp-probe #1'
Test complete
[2026-04-25T23:31:30Z] INFO: ARP resolution succeeded (got response from peer)
[2026-04-25T23:31:30Z] INFO: Test: udp_send_receive
[2026-04-25T23:31:31Z] INFO: UDP send/receive succeeded
[2026-04-25T23:31:31Z] INFO: Test: echo_roundtrip
[2026-04-25T23:31:33Z] INFO: Echo roundtrip succeeded: 5/5 responses received
[2026-04-25T23:31:33Z] INFO: Test: payload_integrity
[2026-04-25T23:31:34Z] INFO: Response received, checking payload match...
[2026-04-25T23:31:34Z] INFO: Payload integrity verified (found in response)
[2026-04-25T23:31:34Z] INFO: Test: jumbo_diagnostics
[2026-04-25T23:31:34Z] INFO: === JUMBO FRAME DIAGNOSTICS ===
[2026-04-25T23:31:34Z] INFO: Interface MTU:
  1500
  9001
  65536
[2026-04-25T23:31:34Z] INFO:   docker0: MTU=1500
[2026-04-25T23:31:34Z] INFO:   ens5: MTU=9001
[2026-04-25T23:31:34Z] INFO:   lo: MTU=65536
[2026-04-25T23:31:34Z] INFO: Routing table MTU column:
Iface	Destination	Gateway 	Flags	RefCnt	Use	Metric	Mask		MTU	Window	IRTT                                                       
ens5	00000000	0101000A	0003	0	0	512	00000000	0	0	0                                                                             
ens5	0200000A	0101000A	0007	0	0	512	FFFFFFFF	0	0	0                                                                             
ens5	0001000A	00000000	0001	0	0	512	00FFFFFF	0	0	0                                                                             
ens5	0101000A	00000000	0005	0	0	512	FFFFFFFF	0	0	0                                                                             
[2026-04-25T23:31:34Z] INFO: DPDK port config (from echo server log):
[2026-04-25T23:31:34Z] INFO:   (no MTU info in echo log)
[2026-04-25T23:31:34Z] INFO: === END JUMBO DIAGNOSTICS ===
[2026-04-25T23:31:34Z] INFO: Test: jumbo_echo_8000
[2026-04-25T23:31:36Z] INFO: Jumbo output: UDP Test Client
Target: 10.0.1.83:9000
Bind address: 10.0.1.200:0
Payload size: 8000 bytes
Count: 3
EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.200:32768 (MAC: 02:ff:aa:49:4f:df)
Backend: dpdk
Sending packets...
Sent 8000 bytes (binary payload)
Received 8000 bytes from 10.0.1.83:9000 (expected 8000, OK)
Sent 8000 bytes (binary payload)
Received 8000 bytes from 10.0.1.83:9000 (expected 8000, OK)
Sent 8000 bytes (binary payload)
Received 8000 bytes from 10.0.1.83:9000 (expected 8000, OK)
Test complete
[2026-04-25T23:31:36Z] INFO: Jumbo frame echo succeeded: 3/3 responses with correct size
[2026-04-25T23:31:36Z] INFO: JUnit XML written to: /tmp/test-results/tier1-dpdk-echo.xml
[2026-04-25T23:31:36Z] INFO: Tier 1 sender tests complete. Results: /tmp/test-results/tier1-dpdk-echo.xml
[2026-04-25T23:38:34Z] INFO: Test: arp_resolution
UDP Test Client
Target: 10.0.1.83:9000
Bind address: 0.0.0.0:0
Message: 'arp-probe'
Count: 1
EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
TELEMETRY: No legacy callbacks, legacy socket not created
DPDK bind failed (Port init failed: Invalid port ID: 0), falling back to tokio
Backend: tokio
Sending packets...
Sent 12 bytes: 'arp-probe #1'
Received 12 bytes from 10.0.1.83:9000: 'arp-probe #1'
Test complete
[2026-04-25T23:38:34Z] INFO: ARP resolution succeeded (got response from peer)
[2026-04-25T23:38:34Z] INFO: Test: udp_send_receive
[2026-04-25T23:38:35Z] INFO: UDP send/receive succeeded
[2026-04-25T23:38:35Z] INFO: Test: echo_roundtrip
[2026-04-25T23:38:36Z] INFO: Echo roundtrip succeeded: 5/5 responses received
[2026-04-25T23:38:36Z] INFO: Test: payload_integrity
[2026-04-25T23:38:36Z] INFO: Response received, checking payload match...
[2026-04-25T23:38:36Z] INFO: Payload integrity verified (found in response)
[2026-04-25T23:38:36Z] INFO: JUnit XML written to: /tmp/test-results/tier2-kernel-interop.xml
[2026-04-25T23:38:36Z] INFO: Tier 2 sender tests complete. Results: /tmp/test-results/tier2-kernel-interop.xml

receiver-test-client-iperf.log

[2026-04-25T23:43:39Z] INFO: iperf-sends: sent 10 packets, received 10 responses
[2026-04-25T23:43:39Z] INFO: iperf-sends: PASS (sent >= 5 packets)
[2026-04-25T23:43:39Z] INFO: JUnit XML written to: /tmp/test-results/tier3-iperf-sends.xml
[2026-04-25T23:43:39Z] INFO: iperf-sends test complete

sender-test-client-iperf.log

[2026-04-25T23:42:36Z] INFO: Pre-flight: checking DPDK state and ARP cache...
[2026-04-25T23:42:36Z] INFO: Local IP: 10.0.1.200, Peer IP: 10.0.1.83, Port: 9000
[2026-04-25T23:42:36Z] INFO: /proc/net/arp contents:
IP address       HW type     Flags       HW address            Mask     Device
10.0.1.83        0x1         0x2         02:34:43:89:d8:2d     *        ens5
10.0.1.203       0x1         0x2         02:df:8c:93:ae:6d     *        ens5
10.0.1.197       0x1         0x2         02:27:99:17:b3:19     *        ens5
10.0.1.1         0x1         0x2         02:a1:e8:12:e0:87     *        ens5
10.0.1.200       0x1         0x2         02:ff:aa:49:4f:df     *        ens5
[2026-04-25T23:42:36Z] INFO: DPDK runtime state:
No /var/run/dpdk/ directory
[2026-04-25T23:42:36Z] INFO: vfio-pci bindings:
0000:00:06.0
bind
module
new_id
remove_id
uevent
unbind
[2026-04-25T23:42:36Z] INFO: Test binary: /opt/dpdk-stdlib/target/release/test-client
-rwxr-xr-x. 2 root root 1867696 Apr 25 23:29 /opt/dpdk-stdlib/target/release/test-client
[2026-04-25T23:42:36Z] INFO: Launching test-client: /opt/dpdk-stdlib/target/release/test-client --target 10.0.1.83 --port 9000 --bind-ip 10.0.1.200 --count 10 --delay 200
[2026-04-25T23:42:38Z] INFO: Test client output: UDP Test Client
Target: 10.0.1.83:9000
Bind address: 10.0.1.200:0
Message: 'dpdk-to-kernel-test-payload'
Count: 10
EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.200:32768 (MAC: 02:ff:aa:49:4f:df)
Backend: dpdk
Sending packets...
Sent 30 bytes: 'dpdk-to-kernel-test-payload #1'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #1'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #2'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #2'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #3'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #3'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #4'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #4'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #5'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #5'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #6'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #6'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #7'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #7'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #8'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #8'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #9'
Received 30 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #9'
Sent 31 bytes: 'dpdk-to-kernel-test-payload #10'
Received 31 bytes from 10.0.1.83:9000: 'dpdk-to-kernel-test-payload #10'
Test complete
[2026-04-25T23:42:38Z] INFO: our-app-sends: sent 10 packets, received 10 responses
[2026-04-25T23:42:38Z] INFO: our-app-sends: PASS (sent >= 5 packets)
[2026-04-25T23:42:38Z] INFO: JUnit XML written to: /tmp/test-results/tier3-our-app-sends.xml
[2026-04-25T23:42:38Z] INFO: our-app-sends test complete
⚠️ SSM Command Failures (receiver-ssm-failure.log)
=== Polling timeout after 30s ===
Status: InProgress
Instance: i-07a348ac0a0b105a2 (receiver)
Command ID: 34413000-51aa-41f0-b7dd-928a1574892c

=== STDOUT ===


=== STDERR ===


=== Polling timeout after 30s ===
Status: InProgress
Instance: i-07a348ac0a0b105a2 (receiver)
Command ID: b8d92550-cc1e-43ec-aed3-a24b979c2c0f

=== STDOUT ===


=== STDERR ===


=== Polling timeout after 30s ===
Status: InProgress
Instance: i-07a348ac0a0b105a2 (receiver)
Command ID: baa9fcb0-e51f-4a44-9519-10040ab69b48

=== STDOUT ===


=== STDERR ===


=== Polling timeout after 30s ===
Status: InProgress
Instance: i-07a348ac0a0b105a2 (receiver)
Command ID: 01ab3414-e612-4efe-b072-29b068802762

=== STDOUT ===


=== STDERR ===


⚠️ SSM Command Failures (sender-ssm-failure.log)
=== Polling timeout after 30s ===
Status: InProgress
Instance: i-039ef5a00d19ff0ac (sender)
Command ID: adcc762d-98d2-441f-9430-dfde04a46991

=== STDOUT ===


=== STDERR ===


Network & PCI State

receiver-network-interfaces.log

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:57:ab:5e:07:05 brd ff:ff:ff:ff:ff:ff
    altname enp0s5
    altname eni-0a90ebe81af80e860
    altname device-number-0.0
    inet 10.0.1.54/24 metric 512 brd 10.0.1.255 scope global dynamic ens5
       valid_lft 2008sec preferred_lft 2008sec
    inet6 fe80::57:abff:fe5e:705/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:7e:6d:1e:b3 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

sender-network-interfaces.log

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:1e:fc:71:45:49 brd ff:ff:ff:ff:ff:ff
    altname enp0s5
    altname eni-039dd61bcdcd7ae3c
    altname device-number-0.0
    inet 10.0.1.126/24 metric 512 brd 10.0.1.255 scope global dynamic ens5
       valid_lft 2030sec preferred_lft 2030sec
    inet6 fe80::1e:fcff:fe71:4549/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:f0:75:02:3a brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

receiver-networking-diag-baseline.txt

=== NETWORKING DIAGNOSTICS ===
timestamp: 2026-04-25T23:30:49Z
hostname: ip-10-0-1-54.ec2.internal
kernel: 6.1.166-197.305.amzn2023.x86_64

=== DPDK PORT STATUS ===

Network devices using DPDK-compatible driver
============================================
0000:00:06.0 'Elastic Network Adapter (ENA) ec20' drv=vfio-pci unused=ena

Network devices using kernel driver
===================================
0000:00:05.0 'Elastic Network Adapter (ENA) ec20' if=ens5 drv=ena unused=vfio-pci *Active*

No 'Baseband' devices detected
==============================

No 'Crypto' devices detected
============================

No 'DMA' devices detected
=========================

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

No 'Regex' devices detected
===========================

=== IP ADDRESSES ===
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:57:ab:5e:07:05 brd ff:ff:ff:ff:ff:ff
    altname enp0s5
    altname eni-0a90ebe81af80e860
    altname device-number-0.0
    inet 10.0.1.54/24 metric 512 brd 10.0.1.255 scope global dynamic ens5
       valid_lft 3309sec preferred_lft 3309sec
    inet6 fe80::57:abff:fe5e:705/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:7e:6d:1e:b3 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

=== ARP TABLE ===
10.0.1.1 dev ens5 lladdr 02:a1:e8:12:e0:87 REACHABLE 
10.0.1.203 dev ens5 lladdr 02:df:8c:93:ae:6d REACHABLE 

=== ROUTE TABLE ===
default via 10.0.1.1 dev ens5 proto dhcp src 10.0.1.54 metric 512 
10.0.0.2 via 10.0.1.1 dev ens5 proto dhcp src 10.0.1.54 metric 512 
10.0.1.0/24 dev ens5 proto kernel scope link src 10.0.1.54 metric 512 
10.0.1.1 dev ens5 proto dhcp scope link src 10.0.1.54 metric 512 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 

=== IMDS: ENI INFORMATION ===
ENI MACs found: 02:34:43:89:d8:2d/ 02:57:ab:5e:07:05/ 

--- ENI: 02:34:43:89:d8:2d/ ---
  device-number: 1
  local-ipv4s: 10.0.1.83
  subnet-id: subnet-044def3113d5e0099
  subnet-cidr: 10.0.1.0/24

--- ENI: 02:57:ab:5e:07:05/ ---
  device-number: 0
  local-ipv4s: 10.0.1.54
  subnet-id: subnet-044def3113d5e0099
  subnet-cidr: 10.0.1.0/24


=== GATEWAY ARP TEST ===
Gateway IP: 10.0.1.1
Gateway ARP entry:
10.0.1.1 dev ens5 lladdr 02:a1:e8:12:e0:87 REACHABLE 

arping result:
ARPING 10.0.1.1 from 10.0.1.54 ens5
Unicast reply from 10.0.1.1 [02:A1:E8:12:E0:87]  0.535ms
Sent 1 probes (1 broadcast(s))
Received 1 response(s)

=== HUGEPAGE STATUS ===
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
FileHugePages:         0 kB
HugePages_Total:    1024
HugePages_Free:     1024
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:         2097152 kB

=== VFIO STATUS ===
total 0
drwxr-xr-x.  2 root root       80 Apr 25 23:30 .
drwxr-xr-x. 14 root root     3100 Apr 25 23:29 ..
crw-------.  1 root root 243,   0 Apr 25 23:30 noiommu-0
crw-rw-rw-.  1 root root  10, 196 Apr 25 23:25 vfio

noiommu mode:
Y

=== DPDK SHARED MEMORY ===
no /var/run/dpdk/ directory (clean state)

=== DPDK-RELATED DMESG (last 30 lines) ===
[    0.067775] x86/fpu: Enabled xstate features 0x2ff, context size is 2568 bytes, using 'compacted' format.
[    0.073841] audit: type=2000 audit(1777159544.572:1): state=initialized audit_enabled=0 res=1
[    0.076117] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[    0.100314] ACPI: Interpreter enabled
[    0.100314] ACPI: Enabled 2 GPEs in block 00 to 0F
[    0.124692] pci 0000:00:05.0: enabling Extended Tags
[    0.148553] SGI XFS with ACLs, security attributes, quota, no debug enabled
[    0.178067] ACPI: \_SB_.LNKD: Enabled at IRQ 11
[    0.255723] IPI shorthand broadcast: enabled
[    0.255784] AES CTR mode by8 optimization enabled
[    9.185160] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[    9.224117] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[    9.327079] VFIO - User Level meta-driver version: 0.3
[   10.306376] ena 0000:00:05.0: Elastic Network Adapter (ENA) v2.16.1g
[   10.318022] ena 0000:00:05.0: ENA device version: 0.10
[   10.318762] ena 0000:00:05.0: ENA controller version: 0.0.1 implementation version 1
[   10.414997] ena 0000:00:05.0: ENA Large LLQ is disabled
[   10.426780] ena 0000:00:05.0: Elastic Network Adapter (ENA) found at mem c0500000, mac addr 02:57:ab:5e:07:05
[   10.515038] ena 0000:00:05.0 ens5: renamed from eth0
[   12.421737] ena 0000:00:05.0 ens5: Local page cache is disabled for less than 16 channels
[  250.151044] pci 0000:00:06.0: enabling Extended Tags
[  250.154892] ena 0000:00:06.0: enabling device (0000 -> 0002)
[  250.167907] ena 0000:00:06.0: ENA device version: 0.10
[  250.168681] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  250.269859] ena 0000:00:06.0: ENA Large LLQ is disabled
[  250.282494] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:34:43:89:d8:2d
[  250.322895] ena 0000:00:06.0 ens6: renamed from eth0
[  250.756601] ena 0000:00:06.0 ens6: Local page cache is disabled for less than 16 channels
[  291.136534] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  291.137846] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device

=== DPDK-RELATED PROCESSES ===
no DPDK processes running

=== END DIAGNOSTICS ===

sender-networking-diag-baseline.txt

=== NETWORKING DIAGNOSTICS ===
timestamp: 2026-04-25T23:30:41Z
hostname: ip-10-0-1-126.ec2.internal
kernel: 6.1.166-197.305.amzn2023.x86_64

=== DPDK PORT STATUS ===

Network devices using DPDK-compatible driver
============================================
0000:00:06.0 'Elastic Network Adapter (ENA) ec20' drv=vfio-pci unused=ena

Network devices using kernel driver
===================================
0000:00:05.0 'Elastic Network Adapter (ENA) ec20' if=ens5 drv=ena unused=vfio-pci *Active*

No 'Baseband' devices detected
==============================

No 'Crypto' devices detected
============================

No 'DMA' devices detected
=========================

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

No 'Regex' devices detected
===========================

=== IP ADDRESSES ===
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:1e:fc:71:45:49 brd ff:ff:ff:ff:ff:ff
    altname enp0s5
    altname eni-039dd61bcdcd7ae3c
    altname device-number-0.0
    inet 10.0.1.126/24 metric 512 brd 10.0.1.255 scope global dynamic ens5
       valid_lft 3316sec preferred_lft 3316sec
    inet6 fe80::1e:fcff:fe71:4549/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:f0:75:02:3a brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

=== ARP TABLE ===
10.0.1.203 dev ens5 lladdr 02:df:8c:93:ae:6d REACHABLE 
10.0.1.1 dev ens5 lladdr 02:a1:e8:12:e0:87 REACHABLE 

=== ROUTE TABLE ===
default via 10.0.1.1 dev ens5 proto dhcp src 10.0.1.126 metric 512 
10.0.0.2 via 10.0.1.1 dev ens5 proto dhcp src 10.0.1.126 metric 512 
10.0.1.0/24 dev ens5 proto kernel scope link src 10.0.1.126 metric 512 
10.0.1.1 dev ens5 proto dhcp scope link src 10.0.1.126 metric 512 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 

=== IMDS: ENI INFORMATION ===
ENI MACs found: 02:1e:fc:71:45:49/ 02:ff:aa:49:4f:df/ 

--- ENI: 02:1e:fc:71:45:49/ ---
  device-number: 0
  local-ipv4s: 10.0.1.126
  subnet-id: subnet-044def3113d5e0099
  subnet-cidr: 10.0.1.0/24

--- ENI: 02:ff:aa:49:4f:df/ ---
  device-number: 1
  local-ipv4s: 10.0.1.200
  subnet-id: subnet-044def3113d5e0099
  subnet-cidr: 10.0.1.0/24


=== GATEWAY ARP TEST ===
Gateway IP: 10.0.1.1
Gateway ARP entry:
10.0.1.1 dev ens5 lladdr 02:a1:e8:12:e0:87 REACHABLE 

arping result:
ARPING 10.0.1.1 from 10.0.1.126 ens5
Unicast reply from 10.0.1.1 [02:A1:E8:12:E0:87]  0.536ms
Sent 1 probes (1 broadcast(s))
Received 1 response(s)

=== HUGEPAGE STATUS ===
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
FileHugePages:         0 kB
HugePages_Total:    1024
HugePages_Free:     1024
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:         2097152 kB

=== VFIO STATUS ===
total 0
drwxr-xr-x.  2 root root       80 Apr 25 23:30 .
drwxr-xr-x. 14 root root     3100 Apr 25 23:29 ..
crw-------.  1 root root 243,   0 Apr 25 23:30 noiommu-0
crw-rw-rw-.  1 root root  10, 196 Apr 25 23:25 vfio

noiommu mode:
Y

=== DPDK SHARED MEMORY ===
no /var/run/dpdk/ directory (clean state)

=== DPDK-RELATED DMESG (last 30 lines) ===
[    0.069296] x86/fpu: Enabled xstate features 0x2ff, context size is 2568 bytes, using 'compacted' format.
[    0.075484] audit: type=2000 audit(1777159544.004:1): state=initialized audit_enabled=0 res=1
[    0.077891] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[    0.102265] ACPI: Interpreter enabled
[    0.102265] ACPI: Enabled 2 GPEs in block 00 to 0F
[    0.126645] pci 0000:00:05.0: enabling Extended Tags
[    0.149973] SGI XFS with ACLs, security attributes, quota, no debug enabled
[    0.179274] ACPI: \_SB_.LNKD: Enabled at IRQ 11
[    0.261163] IPI shorthand broadcast: enabled
[    0.261225] AES CTR mode by8 optimization enabled
[    9.825270] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[    9.868212] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[    9.933517] VFIO - User Level meta-driver version: 0.3
[   11.185853] ena 0000:00:05.0: Elastic Network Adapter (ENA) v2.16.1g
[   11.195047] ena 0000:00:05.0: ENA device version: 0.10
[   11.195796] ena 0000:00:05.0: ENA controller version: 0.0.1 implementation version 1
[   11.295050] ena 0000:00:05.0: ENA Large LLQ is disabled
[   11.307258] ena 0000:00:05.0: Elastic Network Adapter (ENA) found at mem c0500000, mac addr 02:1e:fc:71:45:49
[   11.425099] ena 0000:00:05.0 ens5: renamed from eth0
[   12.958263] ena 0000:00:05.0 ens5: Local page cache is disabled for less than 16 channels
[  249.716641] pci 0000:00:06.0: enabling Extended Tags
[  249.720486] ena 0000:00:06.0: enabling device (0000 -> 0002)
[  249.733414] ena 0000:00:06.0: ENA device version: 0.10
[  249.734175] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  249.833401] ena 0000:00:06.0: ENA Large LLQ is disabled
[  249.845736] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:ff:aa:49:4f:df
[  249.893615] ena 0000:00:06.0 ens6: renamed from eth0
[  250.317946] ena 0000:00:06.0 ens6: Local page cache is disabled for less than 16 channels
[  284.816449] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  284.817791] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device

=== DPDK-RELATED PROCESSES ===
no DPDK processes running

=== END DIAGNOSTICS ===
⚠️ Crash Diagnostics

receiver-dmesg-crashes.log

[    0.073841] pid_max: default: 32768 minimum: 301
[    0.126783] iommu: Default domain type: Translated 
[    0.127505] NetLabel:  unlabeled traffic allowed by default
[    0.134269] PCI: CLS 0 bytes, default 64
[    0.196492] nvme nvme0: 2/0/0 default/read/poll queues
[    0.422460] systemd[1]: systemd 252.23-11.amzn2023 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 -BZIP2 -LZ4 +XZ +ZLIB -ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[    0.422553] systemd[1]: No hostname configured, using default hostname.
[    0.537600] systemd[1]: Queued start job for default target initrd.target.
[    6.656624] systemd[1]: systemd 252.23-11.amzn2023 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 -BZIP2 -LZ4 +XZ +ZLIB -ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[   18.294106] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[  329.944534] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:6881)
[  754.310379] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:20121)
[ 1464.603290] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:41549)

sender-dmesg-crashes.log

[    0.075484] pid_max: default: 32768 minimum: 301
[    0.128767] iommu: Default domain type: Translated 
[    0.129510] NetLabel:  unlabeled traffic allowed by default
[    0.135484] PCI: CLS 0 bytes, default 64
[    0.198071] nvme nvme0: 2/0/0 default/read/poll queues
[    0.428810] systemd[1]: systemd 252.23-11.amzn2023 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 -BZIP2 -LZ4 +XZ +ZLIB -ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[    0.428904] systemd[1]: No hostname configured, using default hostname.
[    0.548538] systemd[1]: Queued start job for default target initrd.target.
[    7.224111] systemd[1]: systemd 252.23-11.amzn2023 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 -BZIP2 -LZ4 +XZ +ZLIB -ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[   18.876250] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[ 1057.876696] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:35521)
Kernel Console (dmesg)

receiver-console-output.log (PCI/driver events only)

[  250.152825] pci 0000:00:06.0: BAR 2: assigned [mem 0xc0600000-0xc06fffff pref]
[  250.153879] pci 0000:00:06.0: BAR 0: assigned [mem 0xc0508000-0xc050bfff]
[  250.154892] ena 0000:00:06.0: enabling device (0000 -> 0002)
[  250.167907] ena 0000:00:06.0: ENA device version: 0.10
[  250.168681] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  250.269859] ena 0000:00:06.0: ENA Large LLQ is disabled
[  250.282494] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:34:43:89:d8:2d
[  250.322895] ena 0000:00:06.0 ens6: renamed from eth0
[  250.756601] ena 0000:00:06.0 ens6: Local page cache is disabled for less than 16 channels
[  291.136534] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  291.137846] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[  329.944534] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:6881)
[  493.721818] vfio-pci 0000:00:06.0: Removing from iommu group 0
[  494.730404] ena 0000:00:06.0: ENA device version: 0.10
[  494.731190] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  494.825807] ena 0000:00:06.0: ENA Large LLQ is disabled
[  494.838421] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:34:43:89:d8:2d
[  494.848410] ena 0000:00:06.0 eth0: Local page cache is disabled for less than 16 channels
[  627.532917] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  627.534240] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[  754.310379] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:20121)
[  830.401598] vfio-pci 0000:00:06.0: Removing from iommu group 0
[  831.414410] ena 0000:00:06.0: ENA device version: 0.10
[  831.415199] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  831.516405] ena 0000:00:06.0: ENA Large LLQ is disabled
[  831.528928] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:34:43:89:d8:2d
[  831.539291] ena 0000:00:06.0 eth0: Local page cache is disabled for less than 16 channels
[ 1330.065898] vfio-pci 0000:00:06.0: Adding to iommu group 0
[ 1330.067215] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[ 1464.603290] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:41549)

sender-console-output.log (PCI/driver events only)

[  250.317946] ena 0000:00:06.0 ens6: Local page cache is disabled for less than 16 channels
[  284.816449] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  284.817791] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[  345.622666] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:6797)
[  346.432517] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:6901)
[  348.212386] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:7134)
[  349.772434] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:7158)
[  350.562407] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:7218)
[  401.317953] vfio-pci 0000:00:06.0: Removing from iommu group 0
[  402.327883] ena 0000:00:06.0: ENA device version: 0.10
[  402.328677] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  402.427859] ena 0000:00:06.0: ENA Large LLQ is disabled
[  402.440054] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:ff:aa:49:4f:df
[  402.450459] ena 0000:00:06.0 eth0: Local page cache is disabled for less than 16 channels
[  869.664551] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  869.665930] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[ 1012.578363] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:34723)
[ 1057.876696] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:35521)
[ 1120.818202] vfio-pci 0000:00:06.0: Removing from iommu group 0
[ 1121.831697] ena 0000:00:06.0: ENA device version: 0.10
[ 1121.832509] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[ 1121.931694] ena 0000:00:06.0: ENA Large LLQ is disabled
[ 1121.943876] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:ff:aa:49:4f:df
[ 1121.954831] ena 0000:00:06.0 eth0: Local page cache is disabled for less than 16 channels
[ 1255.330697] vfio-pci 0000:00:06.0: Adding to iommu group 0
[ 1255.332182] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[ 1481.141202] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:52791)
[ 1481.891113] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:52831)
[ 1483.641035] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:52902)
[ 1485.400952] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:52973)

@github-actions

Copy link
Copy Markdown

Synthetic Performance Results (run)

Commit: 90d9a07c

✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)

Synthetic UDP Performance Results

Measures framework overhead: sync dpdk_udp::UdpSocket vs async (std::sync::Mutex + try_recv_from).

Test Payload Sync PPS Async PPS Ratio (sync/async) Async ns/op
TX send_to 64B 11.6M 11.1M 1.0x 90
RX recv_from 64B 3.5M 4.6M 0.8x 218
TX send_to 1400B 1.6M 1.6M 1.0x 610
RX recv_from 1400B 1.0M 1.1M 0.9x 892

Avg sync/async ratio: 0.9x, worst: 1.0x

Good: Async wrapper is within 1.0x of sync — minimal framework overhead.

@github-actions

Copy link
Copy Markdown

Synthetic Performance Results — Graviton (run)

Commit: 90d9a07c

✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)
✅ synthetic UDP socket bound to 10.0.0.1:9000 (MAC: 02:00:00:00:00:01)

Synthetic UDP Performance Results

Measures framework overhead: sync dpdk_udp::UdpSocket vs async (std::sync::Mutex + try_recv_from).

Test Payload Sync PPS Async PPS Ratio (sync/async) Async ns/op
TX send_to 64B 11.6M 11.0M 1.1x 90
RX recv_from 64B 3.5M 4.6M 0.7x 215
TX send_to 1400B 1.6M 1.6M 1.0x 611
RX recv_from 1400B 1.1M 1.2M 0.9x 860

Avg sync/async ratio: 0.9x, worst: 1.1x

Good: Async wrapper is within 1.1x of sync — minimal framework overhead.

@gspivey
gspivey merged commit 5a4580f into development Apr 25, 2026
7 checks passed
@gspivey
gspivey deleted the agent/ipv6-header-build-parse branch April 26, 2026 00:00
@github-actions

Copy link
Copy Markdown

[CI] Stage: Deploy

Infrastructure ready.

  • Sender: i-044e48d6ebb4bbd94 (DPDK ENI: 10.0.1.187)
  • Receiver: i-040bcf7427d7decbb (DPDK ENI: 10.0.1.54)
  • Both instances SSM-ready.

@github-actions

Copy link
Copy Markdown

[CI] Stage: Deploy

Infrastructure ready.

  • Sender: i-0e9f5e5ad165bbe4d (DPDK ENI: 10.0.1.10)
  • Receiver: i-08b7ee72e9976b09c (DPDK ENI: 10.0.1.77)
  • Both instances SSM-ready.

@github-actions

Copy link
Copy Markdown

[CI] Stage: Summary

All tests PASSED.

ARP seeding: kernel /proc/net/arp (automatic)

  • tier1-dpdk-echo: 6 tests, 0 failures
  • tier2-kernel-interop: 4 tests, 0 failures
  • tier3-iperf-interop: 1 tests, 0 failures
  • tier3-iperf-interop: 1 tests, 0 failures

1 similar comment
@github-actions

Copy link
Copy Markdown

[CI] Stage: Summary

All tests PASSED.

ARP seeding: kernel /proc/net/arp (automatic)

  • tier1-dpdk-echo: 6 tests, 0 failures
  • tier2-kernel-interop: 4 tests, 0 failures
  • tier3-iperf-interop: 1 tests, 0 failures
  • tier3-iperf-interop: 1 tests, 0 failures

@github-actions

Copy link
Copy Markdown

✅ Integration Tests Passed (Run 24943721363)

Branch: 49/merge | Commit: 90d9a07c

Test Results

  • tier1-dpdk-echo.xml: 6 tests, 0 failures, skipped
  • tier2-kernel-interop.xml: 4 tests, 0 failures, skipped
  • tier3-iperf-sends.xml: 1 tests, 0 failures, skipped
  • tier3-our-app-sends.xml: 1 tests, 0 failures, skipped

Application Logs (last 20 lines)

receiver-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.77:9000 (MAC: 02:7a:48:d4:f6:8b)
echo listening on 10.0.1.77:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.10:9000 (MAC: 02:05:32:78:c3:ed)
echo listening on 10.0.1.10:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-test-client.log

EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
TELEMETRY: No legacy callbacks, legacy socket not created
DPDK bind failed (Port init failed: Invalid port ID: 0), falling back to tokio
Backend: tokio
Sending packets...
Sent 12 bytes: 'arp-probe #1'
Received 12 bytes from 10.0.1.77:9000: 'arp-probe #1'
Test complete
[2026-04-26T00:10:44Z] INFO: ARP resolution succeeded (got response from peer)
[2026-04-26T00:10:44Z] INFO: Test: udp_send_receive
[2026-04-26T00:10:45Z] INFO: UDP send/receive succeeded
[2026-04-26T00:10:45Z] INFO: Test: echo_roundtrip
[2026-04-26T00:10:46Z] INFO: Echo roundtrip succeeded: 5/5 responses received
[2026-04-26T00:10:46Z] INFO: Test: payload_integrity
[2026-04-26T00:10:46Z] INFO: Response received, checking payload match...
[2026-04-26T00:10:46Z] INFO: Payload integrity verified (found in response)
[2026-04-26T00:10:46Z] INFO: JUnit XML written to: /tmp/test-results/tier2-kernel-interop.xml
[2026-04-26T00:10:46Z] INFO: Tier 2 sender tests complete. Results: /tmp/test-results/tier2-kernel-interop.xml

receiver-test-client-iperf.log

[2026-04-26T00:17:14Z] INFO: iperf-sends: sent 10 packets, received 10 responses
[2026-04-26T00:17:14Z] INFO: iperf-sends: PASS (sent >= 5 packets)
[2026-04-26T00:17:14Z] INFO: JUnit XML written to: /tmp/test-results/tier3-iperf-sends.xml
[2026-04-26T00:17:14Z] INFO: iperf-sends test complete

sender-test-client-iperf.log

Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #3'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #4'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #4'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #5'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #5'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #6'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #6'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #7'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #7'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #8'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #8'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #9'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #9'
Sent 31 bytes: 'dpdk-to-kernel-test-payload #10'
Received 31 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #10'
Test complete
[2026-04-26T00:16:13Z] INFO: our-app-sends: sent 10 packets, received 10 responses
[2026-04-26T00:16:13Z] INFO: our-app-sends: PASS (sent >= 5 packets)
[2026-04-26T00:16:13Z] INFO: JUnit XML written to: /tmp/test-results/tier3-our-app-sends.xml
[2026-04-26T00:16:13Z] INFO: our-app-sends test complete
Full Application Logs (last 200 lines each)

receiver-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.77:9000 (MAC: 02:7a:48:d4:f6:8b)
echo listening on 10.0.1.77:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.10:9000 (MAC: 02:05:32:78:c3:ed)
echo listening on 10.0.1.10:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-test-client.log

[2026-04-26T00:07:29Z] INFO: Test: arp_resolution
UDP Test Client
Target: 10.0.1.77:9000
Bind address: 10.0.1.10:0
Message: 'arp-probe'
Count: 1
EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.10:32768 (MAC: 02:05:32:78:c3:ed)
Backend: dpdk
Sending packets...
Sent 12 bytes: 'arp-probe #1'
Received 12 bytes from 10.0.1.77:9000: 'arp-probe #1'
Test complete
[2026-04-26T00:07:30Z] INFO: ARP resolution succeeded (got response from peer)
[2026-04-26T00:07:30Z] INFO: Test: udp_send_receive
[2026-04-26T00:07:32Z] INFO: UDP send/receive succeeded
[2026-04-26T00:07:32Z] INFO: Test: echo_roundtrip
[2026-04-26T00:07:33Z] INFO: Echo roundtrip succeeded: 5/5 responses received
[2026-04-26T00:07:33Z] INFO: Test: payload_integrity
[2026-04-26T00:07:34Z] INFO: Response received, checking payload match...
[2026-04-26T00:07:34Z] INFO: Payload integrity verified (found in response)
[2026-04-26T00:07:34Z] INFO: Test: jumbo_diagnostics
[2026-04-26T00:07:34Z] INFO: === JUMBO FRAME DIAGNOSTICS ===
[2026-04-26T00:07:34Z] INFO: Interface MTU:
  1500
  9001
  65536
[2026-04-26T00:07:34Z] INFO:   docker0: MTU=1500
[2026-04-26T00:07:34Z] INFO:   ens5: MTU=9001
[2026-04-26T00:07:34Z] INFO:   lo: MTU=65536
[2026-04-26T00:07:34Z] INFO: Routing table MTU column:
Iface	Destination	Gateway 	Flags	RefCnt	Use	Metric	Mask		MTU	Window	IRTT                                                       
ens5	00000000	0101000A	0003	0	0	512	00000000	0	0	0                                                                             
ens5	0200000A	0101000A	0007	0	0	512	FFFFFFFF	0	0	0                                                                             
ens5	0001000A	00000000	0001	0	0	512	00FFFFFF	0	0	0                                                                             
ens5	0101000A	00000000	0005	0	0	512	FFFFFFFF	0	0	0                                                                             
[2026-04-26T00:07:34Z] INFO: DPDK port config (from echo server log):
[2026-04-26T00:07:34Z] INFO:   (no MTU info in echo log)
[2026-04-26T00:07:34Z] INFO: === END JUMBO DIAGNOSTICS ===
[2026-04-26T00:07:34Z] INFO: Test: jumbo_echo_8000
[2026-04-26T00:07:36Z] INFO: Jumbo output: UDP Test Client
Target: 10.0.1.77:9000
Bind address: 10.0.1.10:0
Payload size: 8000 bytes
Count: 3
EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.10:32768 (MAC: 02:05:32:78:c3:ed)
Backend: dpdk
Sending packets...
Sent 8000 bytes (binary payload)
Received 8000 bytes from 10.0.1.77:9000 (expected 8000, OK)
Sent 8000 bytes (binary payload)
Received 8000 bytes from 10.0.1.77:9000 (expected 8000, OK)
Sent 8000 bytes (binary payload)
Received 8000 bytes from 10.0.1.77:9000 (expected 8000, OK)
Test complete
[2026-04-26T00:07:36Z] INFO: Jumbo frame echo succeeded: 3/3 responses with correct size
[2026-04-26T00:07:36Z] INFO: JUnit XML written to: /tmp/test-results/tier1-dpdk-echo.xml
[2026-04-26T00:07:36Z] INFO: Tier 1 sender tests complete. Results: /tmp/test-results/tier1-dpdk-echo.xml
[2026-04-26T00:10:44Z] INFO: Test: arp_resolution
UDP Test Client
Target: 10.0.1.77:9000
Bind address: 0.0.0.0:0
Message: 'arp-probe'
Count: 1
EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
TELEMETRY: No legacy callbacks, legacy socket not created
DPDK bind failed (Port init failed: Invalid port ID: 0), falling back to tokio
Backend: tokio
Sending packets...
Sent 12 bytes: 'arp-probe #1'
Received 12 bytes from 10.0.1.77:9000: 'arp-probe #1'
Test complete
[2026-04-26T00:10:44Z] INFO: ARP resolution succeeded (got response from peer)
[2026-04-26T00:10:44Z] INFO: Test: udp_send_receive
[2026-04-26T00:10:45Z] INFO: UDP send/receive succeeded
[2026-04-26T00:10:45Z] INFO: Test: echo_roundtrip
[2026-04-26T00:10:46Z] INFO: Echo roundtrip succeeded: 5/5 responses received
[2026-04-26T00:10:46Z] INFO: Test: payload_integrity
[2026-04-26T00:10:46Z] INFO: Response received, checking payload match...
[2026-04-26T00:10:46Z] INFO: Payload integrity verified (found in response)
[2026-04-26T00:10:46Z] INFO: JUnit XML written to: /tmp/test-results/tier2-kernel-interop.xml
[2026-04-26T00:10:46Z] INFO: Tier 2 sender tests complete. Results: /tmp/test-results/tier2-kernel-interop.xml

receiver-test-client-iperf.log

[2026-04-26T00:17:14Z] INFO: iperf-sends: sent 10 packets, received 10 responses
[2026-04-26T00:17:14Z] INFO: iperf-sends: PASS (sent >= 5 packets)
[2026-04-26T00:17:14Z] INFO: JUnit XML written to: /tmp/test-results/tier3-iperf-sends.xml
[2026-04-26T00:17:14Z] INFO: iperf-sends test complete

sender-test-client-iperf.log

[2026-04-26T00:16:10Z] INFO: Pre-flight: checking DPDK state and ARP cache...
[2026-04-26T00:16:10Z] INFO: Local IP: 10.0.1.10, Peer IP: 10.0.1.77, Port: 9000
[2026-04-26T00:16:10Z] INFO: /proc/net/arp contents:
IP address       HW type     Flags       HW address            Mask     Device
10.0.1.6         0x1         0x2         02:21:99:e0:1c:a3     *        ens5
10.0.1.77        0x1         0x2         02:7a:48:d4:f6:8b     *        ens5
10.0.1.1         0x1         0x2         02:9f:6f:8d:cd:01     *        ens5
10.0.1.10        0x1         0x2         02:05:32:78:c3:ed     *        ens5
[2026-04-26T00:16:10Z] INFO: DPDK runtime state:
No /var/run/dpdk/ directory
[2026-04-26T00:16:10Z] INFO: vfio-pci bindings:
0000:00:06.0
bind
module
new_id
remove_id
uevent
unbind
[2026-04-26T00:16:10Z] INFO: Test binary: /opt/dpdk-stdlib/target/release/test-client
-rwxr-xr-x. 2 root root 1867696 Apr 26 00:05 /opt/dpdk-stdlib/target/release/test-client
[2026-04-26T00:16:10Z] INFO: Launching test-client: /opt/dpdk-stdlib/target/release/test-client --target 10.0.1.77 --port 9000 --bind-ip 10.0.1.10 --count 10 --delay 200
[2026-04-26T00:16:13Z] INFO: Test client output: UDP Test Client
Target: 10.0.1.77:9000
Bind address: 10.0.1.10:0
Message: 'dpdk-to-kernel-test-payload'
Count: 10
EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.10:32768 (MAC: 02:05:32:78:c3:ed)
Backend: dpdk
Sending packets...
Sent 30 bytes: 'dpdk-to-kernel-test-payload #1'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #1'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #2'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #2'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #3'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #3'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #4'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #4'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #5'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #5'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #6'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #6'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #7'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #7'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #8'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #8'
Sent 30 bytes: 'dpdk-to-kernel-test-payload #9'
Received 30 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #9'
Sent 31 bytes: 'dpdk-to-kernel-test-payload #10'
Received 31 bytes from 10.0.1.77:9000: 'dpdk-to-kernel-test-payload #10'
Test complete
[2026-04-26T00:16:13Z] INFO: our-app-sends: sent 10 packets, received 10 responses
[2026-04-26T00:16:13Z] INFO: our-app-sends: PASS (sent >= 5 packets)
[2026-04-26T00:16:13Z] INFO: JUnit XML written to: /tmp/test-results/tier3-our-app-sends.xml
[2026-04-26T00:16:13Z] INFO: our-app-sends test complete
⚠️ SSM Command Failures (receiver-ssm-failure.log)
=== Polling timeout after 30s ===
Status: InProgress
Instance: i-08b7ee72e9976b09c (receiver)
Command ID: 1bda2741-dff9-4399-b2c0-2e5c99c60cf7

=== STDOUT ===


=== STDERR ===


=== Polling timeout after 30s ===
Status: InProgress
Instance: i-08b7ee72e9976b09c (receiver)
Command ID: 2ec9a3bb-4a72-48b9-8dbe-16e749b664d5

=== STDOUT ===


=== STDERR ===


=== Polling timeout after 30s ===
Status: InProgress
Instance: i-08b7ee72e9976b09c (receiver)
Command ID: 334c662e-39ed-4641-8d49-1b8e0dcadfd2

=== STDOUT ===


=== STDERR ===


=== Polling timeout after 30s ===
Status: InProgress
Instance: i-08b7ee72e9976b09c (receiver)
Command ID: 4a6d7138-903f-440b-aac6-8408384dcda0

=== STDOUT ===


=== STDERR ===


⚠️ SSM Command Failures (sender-ssm-failure.log)
=== Polling timeout after 30s ===
Status: InProgress
Instance: i-0e9f5e5ad165bbe4d (sender)
Command ID: 41bb10b4-792d-4f62-b187-6bae0d07cf32

=== STDOUT ===


=== STDERR ===


Network & PCI State

receiver-network-interfaces.log

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:93:58:59:93:63 brd ff:ff:ff:ff:ff:ff
    altname enp0s5
    altname eni-03b39f5bd4a92e503
    altname device-number-0.0
    inet 10.0.1.141/24 metric 512 brd 10.0.1.255 scope global dynamic ens5
       valid_lft 2132sec preferred_lft 2132sec
    inet6 fe80::93:58ff:fe59:9363/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:9e:5d:33:2a brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

sender-network-interfaces.log

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:fe:11:02:11:ab brd ff:ff:ff:ff:ff:ff
    altname enp0s5
    altname eni-051600e45a611a660
    altname device-number-0.0
    inet 10.0.1.112/24 metric 512 brd 10.0.1.255 scope global dynamic ens5
       valid_lft 2155sec preferred_lft 2155sec
    inet6 fe80::fe:11ff:fe02:11ab/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:2c:f9:b0:29 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

receiver-networking-diag-baseline.txt

=== NETWORKING DIAGNOSTICS ===
timestamp: 2026-04-26T00:06:49Z
hostname: ip-10-0-1-141.ec2.internal
kernel: 6.1.166-197.305.amzn2023.x86_64

=== DPDK PORT STATUS ===

Network devices using DPDK-compatible driver
============================================
0000:00:06.0 'Elastic Network Adapter (ENA) ec20' drv=vfio-pci unused=ena

Network devices using kernel driver
===================================
0000:00:05.0 'Elastic Network Adapter (ENA) ec20' if=ens5 drv=ena unused=vfio-pci *Active*

No 'Baseband' devices detected
==============================

No 'Crypto' devices detected
============================

No 'DMA' devices detected
=========================

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

No 'Regex' devices detected
===========================

=== IP ADDRESSES ===
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:93:58:59:93:63 brd ff:ff:ff:ff:ff:ff
    altname enp0s5
    altname eni-03b39f5bd4a92e503
    altname device-number-0.0
    inet 10.0.1.141/24 metric 512 brd 10.0.1.255 scope global dynamic ens5
       valid_lft 3287sec preferred_lft 3287sec
    inet6 fe80::93:58ff:fe59:9363/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:9e:5d:33:2a brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

=== ARP TABLE ===
10.0.1.108 dev ens5 lladdr 02:1d:24:a7:3c:eb STALE 
10.0.1.1 dev ens5 lladdr 02:9f:6f:8d:cd:01 REACHABLE 
10.0.1.6 dev ens5 lladdr 02:21:99:e0:1c:a3 REACHABLE 

=== ROUTE TABLE ===
default via 10.0.1.1 dev ens5 proto dhcp src 10.0.1.141 metric 512 
10.0.0.2 via 10.0.1.1 dev ens5 proto dhcp src 10.0.1.141 metric 512 
10.0.1.0/24 dev ens5 proto kernel scope link src 10.0.1.141 metric 512 
10.0.1.1 dev ens5 proto dhcp scope link src 10.0.1.141 metric 512 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 

=== IMDS: ENI INFORMATION ===
ENI MACs found: 02:7a:48:d4:f6:8b/ 02:93:58:59:93:63/ 

--- ENI: 02:7a:48:d4:f6:8b/ ---
  device-number: 1
  local-ipv4s: 10.0.1.77
  subnet-id: subnet-09085d8a680ca7bd8
  subnet-cidr: 10.0.1.0/24

--- ENI: 02:93:58:59:93:63/ ---
  device-number: 0
  local-ipv4s: 10.0.1.141
  subnet-id: subnet-09085d8a680ca7bd8
  subnet-cidr: 10.0.1.0/24


=== GATEWAY ARP TEST ===
Gateway IP: 10.0.1.1
Gateway ARP entry:
10.0.1.1 dev ens5 lladdr 02:9f:6f:8d:cd:01 REACHABLE 

arping result:
ARPING 10.0.1.1 from 10.0.1.141 ens5
Unicast reply from 10.0.1.1 [02:9F:6F:8D:CD:01]  0.532ms
Sent 1 probes (1 broadcast(s))
Received 1 response(s)

=== HUGEPAGE STATUS ===
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
FileHugePages:         0 kB
HugePages_Total:    1024
HugePages_Free:     1024
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:         2097152 kB

=== VFIO STATUS ===
total 0
drwxr-xr-x.  2 root root       80 Apr 26 00:06 .
drwxr-xr-x. 14 root root     3100 Apr 26 00:05 ..
crw-------.  1 root root 243,   0 Apr 26 00:06 noiommu-0
crw-rw-rw-.  1 root root  10, 196 Apr 26 00:01 vfio

noiommu mode:
Y

=== DPDK SHARED MEMORY ===
no /var/run/dpdk/ directory (clean state)

=== DPDK-RELATED DMESG (last 30 lines) ===
[    0.068485] x86/fpu: Enabled xstate features 0x2ff, context size is 2568 bytes, using 'compacted' format.
[    0.074539] audit: type=2000 audit(1777161686.518:1): state=initialized audit_enabled=0 res=1
[    0.077023] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[    0.101259] ACPI: Interpreter enabled
[    0.101259] ACPI: Enabled 2 GPEs in block 00 to 0F
[    0.125718] pci 0000:00:05.0: enabling Extended Tags
[    0.149466] SGI XFS with ACLs, security attributes, quota, no debug enabled
[    0.179026] ACPI: \_SB_.LNKD: Enabled at IRQ 11
[    0.258201] IPI shorthand broadcast: enabled
[    0.258265] AES CTR mode by8 optimization enabled
[    5.705179] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[    5.743253] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[    5.809772] VFIO - User Level meta-driver version: 0.3
[    6.855536] ena 0000:00:05.0: Elastic Network Adapter (ENA) v2.16.1g
[    6.871032] ena 0000:00:05.0: ENA device version: 0.10
[    6.871793] ena 0000:00:05.0: ENA controller version: 0.0.1 implementation version 1
[    6.982949] ena 0000:00:05.0: ENA Large LLQ is disabled
[    6.996317] ena 0000:00:05.0: Elastic Network Adapter (ENA) found at mem c0500000, mac addr 02:93:58:59:93:63
[    7.195059] ena 0000:00:05.0 ens5: renamed from eth0
[    8.514894] ena 0000:00:05.0 ens5: Local page cache is disabled for less than 16 channels
[  270.666008] pci 0000:00:06.0: enabling Extended Tags
[  270.669861] ena 0000:00:06.0: enabling device (0000 -> 0002)
[  270.678419] ena 0000:00:06.0: ENA device version: 0.10
[  270.679192] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  270.775741] ena 0000:00:06.0: ENA Large LLQ is disabled
[  270.788361] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:7a:48:d4:f6:8b
[  270.828493] ena 0000:00:06.0 ens6: renamed from eth0
[  271.262301] ena 0000:00:06.0 ens6: Local page cache is disabled for less than 16 channels
[  308.964379] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  308.965696] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device

=== DPDK-RELATED PROCESSES ===
no DPDK processes running

=== END DIAGNOSTICS ===

sender-networking-diag-baseline.txt

=== NETWORKING DIAGNOSTICS ===
timestamp: 2026-04-26T00:06:41Z
hostname: ip-10-0-1-112.ec2.internal
kernel: 6.1.166-197.305.amzn2023.x86_64

=== DPDK PORT STATUS ===

Network devices using DPDK-compatible driver
============================================
0000:00:06.0 'Elastic Network Adapter (ENA) ec20' drv=vfio-pci unused=ena

Network devices using kernel driver
===================================
0000:00:05.0 'Elastic Network Adapter (ENA) ec20' if=ens5 drv=ena unused=vfio-pci *Active*

No 'Baseband' devices detected
==============================

No 'Crypto' devices detected
============================

No 'DMA' devices detected
=========================

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

No 'Regex' devices detected
===========================

=== IP ADDRESSES ===
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:fe:11:02:11:ab brd ff:ff:ff:ff:ff:ff
    altname enp0s5
    altname eni-051600e45a611a660
    altname device-number-0.0
    inet 10.0.1.112/24 metric 512 brd 10.0.1.255 scope global dynamic ens5
       valid_lft 3294sec preferred_lft 3294sec
    inet6 fe80::fe:11ff:fe02:11ab/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:2c:f9:b0:29 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

=== ARP TABLE ===
10.0.1.6 dev ens5 lladdr 02:21:99:e0:1c:a3 REACHABLE 
10.0.1.1 dev ens5 lladdr 02:9f:6f:8d:cd:01 REACHABLE 

=== ROUTE TABLE ===
default via 10.0.1.1 dev ens5 proto dhcp src 10.0.1.112 metric 512 
10.0.0.2 via 10.0.1.1 dev ens5 proto dhcp src 10.0.1.112 metric 512 
10.0.1.0/24 dev ens5 proto kernel scope link src 10.0.1.112 metric 512 
10.0.1.1 dev ens5 proto dhcp scope link src 10.0.1.112 metric 512 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 

=== IMDS: ENI INFORMATION ===
ENI MACs found: 02:05:32:78:c3:ed/ 02:fe:11:02:11:ab/ 

--- ENI: 02:05:32:78:c3:ed/ ---
  device-number: 1
  local-ipv4s: 10.0.1.10
  subnet-id: subnet-09085d8a680ca7bd8
  subnet-cidr: 10.0.1.0/24

--- ENI: 02:fe:11:02:11:ab/ ---
  device-number: 0
  local-ipv4s: 10.0.1.112
  subnet-id: subnet-09085d8a680ca7bd8
  subnet-cidr: 10.0.1.0/24


=== GATEWAY ARP TEST ===
Gateway IP: 10.0.1.1
Gateway ARP entry:
10.0.1.1 dev ens5 lladdr 02:9f:6f:8d:cd:01 REACHABLE 

arping result:
ARPING 10.0.1.1 from 10.0.1.112 ens5
Unicast reply from 10.0.1.1 [02:9F:6F:8D:CD:01]  0.541ms
Sent 1 probes (1 broadcast(s))
Received 1 response(s)

=== HUGEPAGE STATUS ===
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
FileHugePages:         0 kB
HugePages_Total:    1024
HugePages_Free:     1024
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:         2097152 kB

=== VFIO STATUS ===
total 0
drwxr-xr-x.  2 root root       80 Apr 26 00:06 .
drwxr-xr-x. 14 root root     3100 Apr 26 00:05 ..
crw-------.  1 root root 243,   0 Apr 26 00:06 noiommu-0
crw-rw-rw-.  1 root root  10, 196 Apr 26 00:01 vfio

noiommu mode:
Y

=== DPDK SHARED MEMORY ===
no /var/run/dpdk/ directory (clean state)

=== DPDK-RELATED DMESG (last 30 lines) ===
[    0.068973] x86/fpu: Enabled xstate features 0x2ff, context size is 2568 bytes, using 'compacted' format.
[    0.075258] audit: type=2000 audit(1777161686.449:1): state=initialized audit_enabled=0 res=1
[    0.077703] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[    0.102238] ACPI: Interpreter enabled
[    0.102238] ACPI: Enabled 2 GPEs in block 00 to 0F
[    0.126076] pci 0000:00:05.0: enabling Extended Tags
[    0.150227] SGI XFS with ACLs, security attributes, quota, no debug enabled
[    0.187113] ACPI: \_SB_.LNKD: Enabled at IRQ 11
[    0.259117] IPI shorthand broadcast: enabled
[    0.259183] AES CTR mode by8 optimization enabled
[    5.285227] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[    5.324775] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[    5.430441] VFIO - User Level meta-driver version: 0.3
[    6.959196] ena 0000:00:05.0: Elastic Network Adapter (ENA) v2.16.1g
[    6.970468] ena 0000:00:05.0: ENA device version: 0.10
[    6.971254] ena 0000:00:05.0: ENA controller version: 0.0.1 implementation version 1
[    7.072495] ena 0000:00:05.0: ENA Large LLQ is disabled
[    7.085621] ena 0000:00:05.0: Elastic Network Adapter (ENA) found at mem c0500000, mac addr 02:fe:11:02:11:ab
[    7.265089] ena 0000:00:05.0 ens5: renamed from eth0
[    8.297569] ena 0000:00:05.0 ens5: Local page cache is disabled for less than 16 channels
[  270.903396] pci 0000:00:06.0: enabling Extended Tags
[  270.907404] ena 0000:00:06.0: enabling device (0000 -> 0002)
[  270.920640] ena 0000:00:06.0: ENA device version: 0.10
[  270.921418] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  271.007717] ena 0000:00:06.0: ENA Large LLQ is disabled
[  271.017214] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:05:32:78:c3:ed
[  271.062928] ena 0000:00:06.0 ens6: renamed from eth0
[  271.529950] ena 0000:00:06.0 ens6: Local page cache is disabled for less than 16 channels
[  301.998379] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  301.999703] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device

=== DPDK-RELATED PROCESSES ===
no DPDK processes running

=== END DIAGNOSTICS ===
⚠️ Crash Diagnostics

receiver-dmesg-crashes.log

[    0.074524] pid_max: default: 32768 minimum: 301
[    0.127828] iommu: Default domain type: Translated 
[    0.128549] NetLabel:  unlabeled traffic allowed by default
[    0.134524] PCI: CLS 0 bytes, default 64
[    0.201629] nvme nvme0: 2/0/0 default/read/poll queues
[    0.425135] systemd[1]: systemd 252.23-11.amzn2023 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 -BZIP2 -LZ4 +XZ +ZLIB -ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[    0.425233] systemd[1]: No hostname configured, using default hostname.
[    0.507892] systemd[1]: Queued start job for default target initrd.target.
[    4.046032] systemd[1]: systemd 252.23-11.amzn2023 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 -BZIP2 -LZ4 +XZ +ZLIB -ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[   13.352125] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[  348.621062] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:6848)
[  542.151321] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:10175)
[ 1334.012370] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:39570)

sender-dmesg-crashes.log

[    0.075236] pid_max: default: 32768 minimum: 301
[    0.128186] iommu: Default domain type: Translated 
[    0.128912] NetLabel:  unlabeled traffic allowed by default
[    0.135236] PCI: CLS 0 bytes, default 64
[    0.206627] nvme nvme0: 2/0/0 default/read/poll queues
[    0.426752] systemd[1]: systemd 252.23-11.amzn2023 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 -BZIP2 -LZ4 +XZ +ZLIB -ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[    0.426849] systemd[1]: No hostname configured, using default hostname.
[    0.518182] systemd[1]: Queued start job for default target initrd.target.
[    3.900986] systemd[1]: systemd 252.23-11.amzn2023 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 -BZIP2 -LZ4 +XZ +ZLIB -ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[   13.417838] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[  930.019703] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:30221)
Kernel Console (dmesg)

receiver-console-output.log (PCI/driver events only)

[  270.668866] pci 0000:00:06.0: BAR 0: assigned [mem 0xc0508000-0xc050bfff]
[  270.669861] ena 0000:00:06.0: enabling device (0000 -> 0002)
[  270.678419] ena 0000:00:06.0: ENA device version: 0.10
[  270.679192] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  270.775741] ena 0000:00:06.0: ENA Large LLQ is disabled
[  270.788361] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:7a:48:d4:f6:8b
[  270.828493] ena 0000:00:06.0 ens6: renamed from eth0
[  271.262301] ena 0000:00:06.0 ens6: Local page cache is disabled for less than 16 channels
[  308.964379] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  308.965696] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[  348.621062] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:6848)
[  489.109214] vfio-pci 0000:00:06.0: Removing from iommu group 0
[  490.118718] ena 0000:00:06.0: ENA device version: 0.10
[  490.119516] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  490.218765] ena 0000:00:06.0: ENA Large LLQ is disabled
[  490.230668] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:7a:48:d4:f6:8b
[  490.268970] ena 0000:00:06.0 ens6: renamed from eth0
[  490.348396] ena 0000:00:06.0 ens6: Local page cache is disabled for less than 16 channels
[  510.163089] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  510.164414] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[  542.151321] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:10175)
[  618.550134] vfio-pci 0000:00:06.0: Removing from iommu group 0
[  619.558883] ena 0000:00:06.0: ENA device version: 0.10
[  619.559658] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  619.658908] ena 0000:00:06.0: ENA Large LLQ is disabled
[  619.670800] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:7a:48:d4:f6:8b
[  619.680676] ena 0000:00:06.0 eth0: Local page cache is disabled for less than 16 channels
[ 1204.974219] vfio-pci 0000:00:06.0: Adding to iommu group 0
[ 1204.975771] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[ 1334.012370] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:39570)

sender-console-output.log (PCI/driver events only)

[  271.529950] ena 0000:00:06.0 ens6: Local page cache is disabled for less than 16 channels
[  301.998379] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  301.999703] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[  363.584655] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:6846)
[  364.354571] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:6866)
[  366.114570] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:6888)
[  367.674541] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:6911)
[  368.464499] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:6971)
[  419.598617] vfio-pci 0000:00:06.0: Removing from iommu group 0
[  420.611529] ena 0000:00:06.0: ENA device version: 0.10
[  420.612307] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  420.711522] ena 0000:00:06.0: ENA Large LLQ is disabled
[  420.723660] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:05:32:78:c3:ed
[  420.733876] ena 0000:00:06.0 eth0: Local page cache is disabled for less than 16 channels
[  728.842322] vfio-pci 0000:00:06.0: Adding to iommu group 0
[  728.843622] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[  884.390121] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:28069)
[  930.019703] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (echo:30221)
[  993.740540] vfio-pci 0000:00:06.0: Removing from iommu group 0
[  994.753337] ena 0000:00:06.0: ENA device version: 0.10
[  994.754128] ena 0000:00:06.0: ENA controller version: 0.0.1 implementation version 1
[  994.853735] ena 0000:00:06.0: ENA Large LLQ is disabled
[  994.866728] ena 0000:00:06.0: Elastic Network Adapter (ENA) found at mem c0508000, mac addr 02:05:32:78:c3:ed
[  994.877375] ena 0000:00:06.0 eth0: Local page cache is disabled for less than 16 channels
[ 1108.329662] vfio-pci 0000:00:06.0: Adding to iommu group 0
[ 1108.331019] vfio-pci 0000:00:06.0: Adding kernel taint for vfio-noiommu group on device
[ 1350.125990] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:39898)
[ 1350.885904] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:39918)
[ 1352.645913] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:39941)
[ 1354.405877] vfio-pci 0000:00:06.0: vfio-noiommu device opened by user (tokio-rt-worker:39964)

@github-actions

Copy link
Copy Markdown

✅ Integration Tests Passed — Graviton (run)

Branch: 49/merge | Commit: 90d9a07c

Test Results

  • tier1-dpdk-echo.xml: 6 tests, 0 failures
  • tier2-kernel-interop.xml: 4 tests, 0 failures
  • tier3-iperf-sends.xml: 1 tests, 0 failures
  • tier3-our-app-sends.xml: 1 tests, 0 failures

Application Logs (last 20 lines)

receiver-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.54:9000 (MAC: 02:d6:ff:a4:0b:c9)
echo listening on 10.0.1.54:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-echo-server.log

EAL: Detected CPU lcores: 2
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Probe PCI driver: net_ena (1d0f:ec20) device: 0000:00:06.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: Some RX offloads not supported by device (flags: 0x1)
Warning: Some TX offloads not supported by device (flags: 0x1)
✅ DPDK UDP socket bound to 10.0.1.187:9000 (MAC: 02:0a:1d:4d:62:3f)
echo listening on 10.0.1.187:9000 (MTU=9001, max_udp_payload=8973)
Shutting down gracefully...

sender-test-client.log

EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
TELEMETRY: No legacy callbacks, legacy socket not created
DPDK bind failed (Port init failed: Invalid port ID: 0), falling back to tokio
Backend: tokio
Sending packets...
Sent 12 bytes: 'arp-probe #1'
Received 12 bytes from 10.0.1.54:9000: 'arp-probe #1'
Test complete
[2026-04-26T00:14:46Z] INFO: ARP resolution succeeded (got response from peer)
[2026-04-26T00:14:46Z] INFO: Test: udp_send_receive
[2026-04-26T00:14:47Z] INFO: UDP send/receive succeeded
[2026-04-26T00:14:47Z] INFO: Test: echo_roundtrip
[2026-04-26T00:14:48Z] INFO: Echo roundtrip succeeded: 5/5 responses received
[2026-04-26T00:14:48Z] INFO: Test: payload_integrity
[2026-04-26T00:14:48Z] INFO: Response received, checking payload match...
[2026-04-26T00:14:48Z] INFO: Payload integrity verified (found in response)
[2026-04-26T00:14:48Z] INFO: JUnit XML written to: /tmp/test-results/tier2-kernel-interop.xml
[2026-04-26T00:14:48Z] INFO: Tier 2 sender tests complete. Results: /tmp/test-results/tier2-kernel-interop.xml

@gspivey gspivey mentioned this pull request May 21, 2026
gspivey added a commit that referenced this pull request May 21, 2026
## Roadmap item

**Encap: IPv6 outer** — Adds IPv6 outer support to all three
encapsulation protocols (VXLAN, GENEVE, GUE), closing out dual-stack
encap in a single PR.

## Changes

For each protocol (GUE, VXLAN, GENEVE), adds:
- `*Config6` struct with `Ipv6Addr` for remote tunnel endpoint
- `build_*_frame_into_v6()` — frame builder using outer IPv6 header +
mandatory UDP6 checksum (RFC 8200 §8.1)
- `try_decap_*_v6()` — decapsulation for IPv6-outer frames
- `*DecapResult6` with `Ipv6Addr` for outer source IP
- `*_ENCAP_OVERHEAD_V6` constant

Wire format: `[Outer Eth 14B][Outer IPv6 40B][Outer UDP 8B][Protocol
Header][Inner frame]`

## Tests added

41 new unit tests (total: 654, up from 613):
- GUE: 13 tests (config, roundtrip, wire format, checksum, edge cases,
perf)
- VXLAN: 13 tests (config, roundtrip, wire format, checksum, VNI
filtering, perf)
- GENEVE: 15 tests (config, roundtrip, TLV options, wire format,
checksum, VNI filtering, perf)

All synthetic PPS benchmarks assert < 10µs/op.

## Tradeoffs

- Inner payload is IPv4-only for now (matching the existing IPv4-outer
encap). Inner IPv6 will come with the full IPv6 socket support (roadmap
task 3).
- No extension header support in the outer IPv6 — uses Next Header = UDP
directly. Extension headers are not needed for tunnel endpoints.
- The IPv6 outer decap functions do not walk extension headers in the
outer IPv6 header (assumes Next Header = UDP). This matches real-world
tunnel deployments where the outer header is minimal.

## Dependencies satisfied

- IPv6 header build/parse (PR #49) ✓
- UDP6 checksum (same PR) ✓  
- IPv6 offload flags (PR #55) ✓

---------

Co-authored-by: Agent Router <agent@agent-router.dev>
gspivey pushed a commit that referenced this pull request Jun 19, 2026
- Append Run #50 to docs/perf-test-log.md (GH Actions run 27851557307)
- No performance regression vs Run #49
- Mark ROADMAP item #32 as complete (PR #96)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant