timeout or cancelling of debugpy.connect call

[vladkol]

I would like to be able to cancel debugpy.connect call the same or similar way they have it with wait_for_client (timeout is ok too).

It would be useful when connecting from remote code to a local listener, and there is no control on the remote code execution.

[int19h]

We'd have to wire this up through pydevd, looks like.

@fabioz, is it easier to provide a high-level API here, or just expose more control over how the socket is created (to allow specifying timeouts etc)?

[vladkol]

We'd have to wire this up through pydevd, looks like.

@fabioz, is it easier to provide a high-level API here, or just expose more control over how the socket is created (to allow specifying timeouts etc)?

Here is our current workaround (on "stoppable" threads)
https://github.com/vladkol/azure-debug-relay/blob/vladkol/azrelay-lifecycle/azdebugrelay/debugpyman.py#L30

[int19h]

I suspect this doesn't work all that reliably, since the aborted thread does not expect such an exception - so after it bubbles up, the internal state might be invalid.

Also, it won't interrupt in the middle of a syscall - but connect() would ultimately block in the corresponding socket API call, no? Or are we blocking somewhere else in practice?

[vladkol]

I suspect this doesn't work all that reliably, since the aborted thread does not expect such an exception - so after it bubbles up, the internal state might be invalid.

Yes. That's why we don't like it.

Also, it won't interrupt in the middle of a syscall - but connect() would ultimately block in the corresponding socket API call, no? Or are we blocking somewhere else in practice?

It's hard to say where exactly it's blocked. In our scenario, remote code connects (debugpy.connect) to a locally running (and listening) VS Code via an Azure Relay network tunnel. We cannot say that the tunnel is properly established because the relay connection doesn't happen until debugpy actually tries to connect. If the relay is broken or they forgot to start VS Code in listening mode, debugpy.connect never comes back. I can detect the error in Python code, but I cannot make debugpy.connect "restart". Should I simply let that debugpy.connect thread go, and make another one?

[int19h]

This is the code in pydevd that ultimately handles the connection:

debugpy/src/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py

Lines 434 to 471 in 4b32a3c

	def start_client(host, port):
	''' connects to a host/port '''
	pydev_log.info("Connecting to %s:%s", host, port)
	s = socket(AF_INET, SOCK_STREAM)
	# Set TCP keepalive on an open socket.
	# It activates after 1 second (TCP_KEEPIDLE,) of idleness,
	# then sends a keepalive ping once every 3 seconds (TCP_KEEPINTVL),
	# and closes the connection after 5 failed ping (TCP_KEEPCNT), or 15 seconds
	try:
	s.setsockopt(SOL_SOCKET, socket_module.SO_KEEPALIVE, 1)
	except (AttributeError, OSError):
	pass # May not be available everywhere.
	try:
	s.setsockopt(socket_module.IPPROTO_TCP, socket_module.TCP_KEEPIDLE, 1)
	except (AttributeError, OSError):
	pass # May not be available everywhere.
	try:
	s.setsockopt(socket_module.IPPROTO_TCP, socket_module.TCP_KEEPINTVL, 3)
	except (AttributeError, OSError):
	pass # May not be available everywhere.
	try:
	s.setsockopt(socket_module.IPPROTO_TCP, socket_module.TCP_KEEPCNT, 5)
	except (AttributeError, OSError):
	pass # May not be available everywhere.
	try:
	# 10 seconds default timeout
	timeout = int(os.environ.get('PYDEVD_CONNECT_TIMEOUT', 10))
	s.settimeout(timeout)
	s.connect((host, port))
	s.settimeout(None) # no timeout after connected
	pydev_log.info("Connected.")
	return s
	except:
	pydev_log.exception("Could not connect to %s: %s", host, port)
	raise

From what you've describe, it sounds like it could be tripping either on the connect timeout, or possibly on keep-alive (the latter might be the case if the tunnel is set up asynchronously). Try PYDEVD_CONNECT_TIMEOUT to see if it's the former.

[vladkol]

Changing the timeout doesn't help - it actually makes the socket connection, but then the other party doesn't respond.
I think wait_for_ready_to_run makes it wait forever.

[int19h]

The other party would be the debug adapter in this case - can you enable logging for it ("logToFile": true), and see what it says?

[fabioz]

I was taking a look at the code and as noted, there is a 10 seconds default timeout for the socket connection, so, it's probably really stuck in wait_for_ready_to_run , which waits for the configurationDone after all the configurations are sent from the IDE.

So, it's kind of tricky in that the connection did work but then nothing happens afterwards...

I think an additional timeout could be passed as a parameter on this case (and if it fails it needs to close the connection, do a cleanup on what the debugger created and raise a TimeoutError).

Does that sound ok?

[int19h]

I'm still not sure what's actually blocking it. Once pydevd can connect to the adapter, the latter should notify the client, which then starts another debug session via the same adapter - and that's where "configurationDone" should come from.

And in a scenario where you'd use debugpy.connect(), the adapter and the client are communicating over stdio, so there shouldn't be a case of client not being there to notify. Hence why I want to take a look at the adapter logs, to see what exactly prevents this all from working as intended.

[vladkol]

@int19h the adapter on another is side is not communicating back because of Azure Hybrid Connection being dropped due to a missing/wrong/expired key. Think about a proxy server that lets you open a socket, but doesn't let data to flow. It is not a bug or anything like that with debugpy, but we'd like to be able to control the timeout by code because we have very little control over the server interactively.

[int19h]

Ahh, I see. Yes, that sounds reasonable, and what @fabioz proposed should fix it.