Skip to content

microVM mode: artifact upload fails — guest inherits Docker's 127.0.0.11 resolver #230

Description

@adsadawutc

Environment

  • Ubuntu 24.04 VM with nested virtualization (/dev/kvm present)
  • Docker Compose, KVM_ENABLED=true
  • code-interpreter @ latest main

Summary

Under Docker Compose in microVM mode, generated files are never delivered.
Code executes fine, but every artifact upload fails and the file is pruned
from the response. file_server never receives a single request.

Root cause

launcher-entrypoint.sh encodes the runner container's /etc/resolv.conf
and hands it to the guest over the kernel command line. Under Docker that
file contains nameserver 127.0.0.11 — Docker's embedded DNS, which only
exists inside the container's network namespace. In the guest it points at
the guest's own loopback, where nothing listens, so every service name fails
to resolve and the upload surfaces as ConnectionRefused.

Confirmed from inside the sandbox:

import socket; socket.gethostbyname('egress_gateway')
# socket.gaierror: [Errno -2] Name or service not known

Logs

sandbox-runner | {"file":"test.txt","code":"ConnectionRefused",
                  "message":"Unable to connect. Is the computer able to access the url?",
                  "path":"http://egress_gateway:3190/sessions/<redacted>/objects/<redacted>",
                  "msg":"Error uploading file"}
sandbox-runner | {"uploaded":0,"total":1,"msg":"Some files failed to upload"}
sandbox-runner | {"dropped":1,"kept":0,
                  "msg":"Pruned files from response because upload did not reach file_server"}

The gateway is healthy and issues the grant successfully in the same second:

egress_gateway | {"msg":"Egress grant created"}
egress_gateway | {"method":"POST","route":"internal","statusCode":201}

The runner container reaches the gateway fine — only the guest cannot.

Workaround

Address the gateway by IP and pin that IP:

# docker-compose.override.yml
services:
  egress_gateway:
    networks:
      default:
        ipv4_address: 10.0.0.50        # any address in the compose subnet

  sandbox-runner:
    dns:
      - 1.1.1.1                        # a resolver valid outside the netns
    volumes:
      - ./guest-resolv.conf:/etc/resolv.conf:ro
    environment:
      - EGRESS_GATEWAY_URL=http://10.0.0.50:3190
      - SANDBOX_FORWARD_TARGET=10.0.0.50:3190

networks:
  default:
    ipam:
      config:
        - subnet: 10.0.0.0/16

guest-resolv.conf holds one line: nameserver 1.1.1.1. The bind mount is
necessary because dns: is appended to Docker's embedded resolver rather
than replacing it.

Both env vars must match, or secure-startup.ts refuses to start
(SANDBOX_FORWARD_TARGET must point to EGRESS_GATEWAY_URL host:port).

After this, uploads succeed:

file_server    | {"msg":"File uploaded successfully: test.txt"}
sandbox-runner | {"file":"test.txt","size":5,"msg":"Uploaded file"}

Suggested fix

Reject or rewrite loopback nameservers in launcher-entrypoint.sh before
passing SANDBOX_RESOLV_CONF into the guest, or resolve service names to
addresses on the container side first.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions