Repository navigation
Client connectivity not working on Raspbian (aarch64) #237
Description
Describe the Bug
Hi,
I have tried to get the "Zero-Trust Private Access" functionality to work for some time and eventually figured out, that it is my newt which is causing the connection to fail. I believe this might have to do with the architecture of my host, which is ARM64 (aarch64). While this host has some special network setup (it is part of a DMZ), I never had issues with the "Web-based Public Access", and I was able to get the "Zero-Trust Private Access" functionality to work in no time on a x86_64 machine.
On the x86_64 host I can see this line on startup of newt:
INFO: 2026/02/17 14:11:48 Client connectivity setup. Ready to accept connections from clients!
This line is missing from the log on the aarch64 host and when I try to connect the app, I will instead get:
INFO: 2026/02/17 12:57:00 WireGuard device is not initialized
and obviously the App will never complete the connection attempt. This makes me believe that there may be some code which is missing for my architecture, or some other issue with setting up the required wireguard tunnel.
Environment
- OS Type & Version: Debian GNU/Linux 12 (bookworm) (<-- this is Raspbian)
- Kernel Version: 6.12.34+rpt-rpi-v8
- Pangolin Version: v1.15.4
- Gerbil Version: v1.3.0
- Traefik Version: I don't know, :latest 🙃
- Newt Version: v1.9.0
- Docker Version: v28.3.2
To Reproduce
This is my docker-compose.yml on the Raspberry Pi 4 Model B Rev 1.5:
services:
newt:
image: fosrl/newt
container_name: newt
restart: unless-stopped
network_mode: host
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- TZ=Europe/Berlin
- PANGOLIN_ENDPOINT=https://xxxxxxxx
- NEWT_ID=xxxxxxxxxx
- NEWT_SECRET=xxxxxxxxxx
- DOCKER_SOCKET=unix:///var/run/docker.sockExpected Behavior
Same behavior as on x86_64 obviously 🙂
Activity
github-actions commentedon Mar 4, 2026
This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.
hanjo commentedon Mar 4, 2026
In the meantime, Pangolin v1.16.2 and Newt v1.10.2 were released, but I still see the same issue. Any chance to look into this?
Thanks!
LaurenceJJones commentedon Mar 7, 2026
Could you set the environment
LOG_LEVEL=DEBUGand provide the full logs so we can dive deeper. (just ensure to strip any sensitive information from the output)hanjo commentedon Mar 7, 2026
Sure, here you go:
LaurenceJJones commentedon Mar 9, 2026
Thanks for the detailed debug logs @hanjo - they were very helpful in tracking this down.
So my theory so far is a race condition in Pangolin, not a newt client issue. The handleGetConfigMessage handler has two conditions that cause it to silently drop requests without sending a response:
Endpoint not set yet (handleGetConfigMessage.ts:59-64):
if (!existingSite.endpoint) {
logger.debug(
In newt get config: existing site has no endpoint, skipping);return; // No response sent - client times out
}
Hole punch too old (handleGetConfigMessage.ts:70-75):
if (existingSite.lastHolePunch && now - existingSite.lastHolePunch > 5) {
logger.warn(
Site last hole punch is too old, skipping);return; // No response sent - client times out
}
Why it could be affecting ARM64/aarch64 only?
The endpoint and lastHolePunch fields are only set when the hole punch succeeds via Gerbil (updateHolePunch.ts:321-328). On ARM64/Raspberry Pi:
- The 10 retry attempts (20 seconds) have been exhausted, OR
- The lastHolePunch timestamp is now >5 seconds old
Evidence from Logs
Could it be anything else?
Yes in theory maybe the DMZ the device is behind is too strict, but is the
x86device in the same DMZ?Current ideas to fix:
Option A: Increase the 5-second window
The 5-second lastHolePunch requirement is very strict. Increasing to 30 seconds would accommodate slower ARM64 initialization, but also could introduce mismatch if not handled correctly (most stateful firewalls hold routes for 30 seconds maximum)
Option B: Return an error instead of silent drop
Send an explicit error response so the client knows to retry:
Option C: Client-side resilience
Have newt wait for hole punch confirmation before requesting config, or implement longer/infinite retries for get-config.
hanjo commentedon Mar 9, 2026
Wow, very intersting. I wouldn't have figured the performance of the device to be a factor. The x86_64 machine I'm running newt on is a Intel Core i5-9500T which should be about 5-6x times as fast. A more relevant measure is probably the SSD compared to the SD card, which has a huge influence on I/O.
Not sure what Option makes most sense, but if you need me to test some development version, let me know. I'm running everything in docker, if that makes any difference.
github-actions commentedon Mar 24, 2026
This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.
hanjo commentedon Mar 24, 2026
Any news on this issue @LaurenceJJones ?
github-actions commentedon Apr 8, 2026
This issue has been automatically marked as stale due to 14 days of inactivity. It will be closed in 14 days if no further activity occurs.
hanjo commentedon Apr 8, 2026
It seems also with Pangolin v1.17.0 and Newt v1.11.0 this is still not working.
dpurnam commentedon Apr 10, 2026
+1 (AMD based newt sites work fine with private resources but not with ARM based newt-sites)
hanjo commentedon Apr 28, 2026
I just updated to newt v1.12.0 and it seems the situation has not changed (i.e. the issue is still the same).
LaurenceJJones commentedon Apr 28, 2026
Yes apologies me and @oschwartz10612 have a ticket to dive into this but had a few set backs with the release of pangolin.
hanjo commentedon Apr 28, 2026
No worries, I can understand this isn't the most pressing thing to look into. Just trying to keep the bot from closing this issue, before it is resolved 🙃
Thanks for your great work!
LaurenceJJones commentedon Apr 28, 2026
I added a tag now so the bot wont close or mark as stale 👍🏻
oschwartz10612 commentedon Apr 28, 2026
I was actually testing on a arm32v7 Raspberry Pi yesterday and it was working fine. We have newts deployed on arm64 EC2 graviton instances which also work. Must be a specific race condition as maybe Loz is saying. Needs more investigation.
hanjo commentedon Apr 30, 2026
Thanks for looking into this. I now believe my bug report is a false positive. After having re-read the entire issue discussion once more and with the statement, that it works for you, I was checking my configuration once again. In my DMZ I disallow outbound Internet connectivity and I have explicitly allowed tcp/udp 443 and udp 51820 to pangolin. My other newt is in a different network segment without these restrictions. I ran a tcpdump and noticed that newt also tries to connect on udp 21820 to pangolin, which I previously believed only clients (as in "user devices") need to connect to. So I allowed this in the firewall and voilà it works.
Bottom line: it was a misconfiguration on my side; my apologies for having wasted your time.
I leave this issue open in case you would like to use it to trace that race condition, but from my point of view feel free to close it 👍
LaurenceJJones commentedon Apr 30, 2026
Ahhh! makes perfect sense, maybe we should clarify a bit more in the documentation then? that even though we say "Clients" it is the co-ordination port so both sides as in a client and Newt itself need to be able to communicate to it.
hanjo commentedon Apr 30, 2026
Better documentation can never hurt, can it? 🙃 I think port 21820 is only mentioned twice in the documentation. Maybe it would be helpful to mention the communication requirements on the newt documentation page at https://docs.pangolin.net/manage/sites/install-site#running-newt