Drop the Dag inventory from Lang-SDK bundle metadata

.rat-excludes

@@ -328,8 +328,9 @@ www-hash.txt
 # Vendored-in code
 /src/airflow/providers/google/_vendor/*
 
-# TypeScript bundle golden fixture: its bytes are digest-pinned, so no license header can be added
+# TypeScript bundle golden fixtures: their bytes are digest-pinned, so no license header can be added
 /ts-sdk/tests/cli/fixtures/bundle-v1.min.mjs
+/ts-sdk/tests/cli/fixtures/bundle-v1-with-task-handlers.min.mjs
 
 # Java SDK build outputs
 /java-sdk/bin/*

airflow-core/adr/lang-sdk/0014-bundle-metadata-and-cache-digest.md

@@ -119,4 +119,7 @@ something; consumers compare for equality and interpret nothing.
 
 - Dynamic Dag rendering works.
 - The canonical schema drops the identifier mapping entirely, the coordinator task execution side will rely on persisted rel_path instead of discovering the artifact every time.
-- The packer no longer needs to execute the artifact at all. `supervisor_schema_version` is a compile-time constant of the SDK.
+- The Go packer runs nothing. It takes `supervisor_schema_version`, a compile-time constant of the SDK, from its own go-sdk and the SDK version from the binary's build information, and it refuses a binary built against another go-sdk than its own. A cross-built binary packs without a host build, and the bundle binary has no `--airflow-metadata` flag.
+- The TypeScript packer still runs the bundle, for its schema version, the source file of each Dag declared in TypeScript ([ADR-0015](0015-per-dag-source-in-bundle-artifact.md)) and its pack-time checks, but it embeds no `task_handlers`. `dag_source_paths` is the one place a Dag id remains in TypeScript metadata. It is for display, and nothing reads it to find an artifact.
+- A bundle packed before this change may still carry `dags` (Go) or `task_handlers` (TypeScript) in its metadata. Readers ignore them.
+- Java never had an inventory. The Gradle plugin adds `Airflow-Cache-Digest` to the JAR manifest, next to its `Main-Class`, and for a fat JAR `Airflow-Supervisor-Schema-Version` too, which a thin JAR takes from the `airflow-sdk` JAR. It runs nothing. `Airflow-Cache-Digest` marks a handler JAR.

airflow-core/docs/authoring-and-scheduling/language-sdks/go.rst

@@ -28,10 +28,10 @@ to a compiled Go *bundle* that is launched by
 :class:`~airflow.sdk.coordinators.executable.ExecutableCoordinator` for each task instance.
 
 Because Go is a compiled language, every task must be compiled ahead of time and registered inside a single,
-self-contained native executable called a **bundle**. The bundle also embeds its Dag source and a metadata
-manifest (the ``dag_id`` and ``task_id`` map) in a footer appended to the executable, so the executable *is*
-the bundle: one runnable file to ship, with no separate manifest or archive. The
-:ref:`airflow-go-pack <go-sdk/build>` tool builds and packs that bundle.
+self-contained native executable called a **bundle**. The bundle also embeds its Dag source and a small
+metadata manifest (the SDK version and the supervisor schema version it was built against) in a footer
+appended to the executable, so the executable *is* the bundle: one runnable file to ship, with no separate
+manifest or archive. The :ref:`airflow-go-pack <go-sdk/build>` tool builds and packs that bundle.
 
 .. contents:: Contents
    :local:
@@ -136,7 +136,8 @@ Go entry point
 
 Build a bundle with ``airflow.Bundle()``, register a handler for each task, and call ``Serve`` as the last
 statement of ``main``. The ``Register`` calls are the single source of truth for which ``dag_id`` and task
-names this bundle can run, so the generated manifest can never drift from what the binary actually executes.
+names this bundle can run: the Dag processor asks the bundle for them, so it always sees what the binary
+actually executes.
 
 .. code-block:: go
 
@@ -458,6 +459,10 @@ SDKs, specified in :doc:`task-sdk:executable-bundle-spec`.
 to your bundle module's ``go.mod`` and run it with ``go tool airflow-go-pack``. This pins the packer version
 per project.
 
+The packer never runs the binary it packs. It reads the go-sdk version the binary was built against from the
+binary's build information, and refuses a binary built against a different go-sdk version than the packer's,
+so run it from the module that builds the bundle.
+
 Build and pack in one step; any flags after ``--`` are forwarded verbatim to ``go build``:
 
 .. code-block:: bash
@@ -484,20 +489,13 @@ architecture than your build machine (for example, deploying to a Linux host fro
       --output /opt/airflow/go-task-handlers/sample-dag-bundle \
       ./example/bundle
 
-Alternatively, pack a pre-built binary with ``--executable`` / ``--source``. The packer normally execs the
-binary with ``--airflow-metadata`` to read its manifest, but a cross-compiled binary cannot run on the build
-host. In that case, generate the manifest on a machine that *can* run the binary and feed it to the packer
-with ``--airflow-metadata``:
+Alternatively, pack a pre-built binary with ``--executable`` / ``--source``. The packer does not run the
+binary, so one built for any platform packs on any host:
 
 .. code-block:: bash
 
-    # On a linux/amd64 machine:
-    go build -o my-bundle ./example/bundle
-    ./my-bundle --airflow-metadata > airflow-metadata.yaml
-
-    # Back on the darwin/arm64 machine:
-    go tool airflow-go-pack --executable ./my-bundle --source main.go \
-      --airflow-metadata airflow-metadata.yaml
+    GOOS=linux GOARCH=amd64 go build -o my-bundle ./example/bundle
+    go tool airflow-go-pack --executable ./my-bundle --source ./example/bundle/main.go
 
 (``--executable`` is mutually exclusive with ``--goos`` / ``--goarch`` and with ``go build`` flags after
 ``--``, since it packs an already-built binary instead of building one.)

airflow-core/docs/authoring-and-scheduling/language-sdks/index.rst

@@ -264,8 +264,8 @@ coordinator; JVM languages, for example, compile to bytecode that requires a JRE
 To support a new such language, produce a *bundle* in the shared on-disk format the coordinator consumes and
 speak the coordinator IPC protocol (the ``--comm`` / ``--logs`` socket arguments). That format - the
 ``AFBNDL01`` footer appended to the executable, the binary integrity hash, and the ``airflow-metadata.yaml``
-manifest of ``dag_id``\ s and ``task_id``\ s - is specified, together with the reader algorithm and the
-compatibility/versioning rules, in :doc:`task-sdk:executable-bundle-spec`. That page also publishes a
-machine-readable JSON Schema for the manifest, for use by build tooling and validators. Follow the spec to
-make a new language's bundles discoverable by Airflow with no change to the scheduler, worker, or UI; the
-:doc:`Go SDK <go>` is a worked reference implementation.
+manifest - is specified, together with the reader algorithm and the compatibility/versioning rules, in
+:doc:`task-sdk:executable-bundle-spec`. That page also publishes a machine-readable JSON Schema for the
+manifest, for use by build tooling and validators. Follow the spec to make a new language's bundles
+discoverable by Airflow with no change to the scheduler, worker, or UI; the :doc:`Go SDK <go>` is a worked
+reference implementation.

airflow-core/docs/authoring-and-scheduling/language-sdks/typescript.rst

@@ -493,10 +493,11 @@ Building and packaging
 ----------------------
 
 ``airflow-ts-pack`` (shipped with the SDK) bundles the entry module and all of its imports with esbuild into
-a single self-contained, minified ESM file, ``bundle.min.mjs``, and embeds the manifest (the ``dag_id`` and
-``task_id`` map plus the supervisor schema version) after a leading compact JSON ``//# airflowBundle=...``
-layout header. The layout records the byte ranges and SHA-256 digests of the manifest and executable code,
-so there is one file to deploy, with no separate manifest or ``node_modules``.
+a single self-contained, minified ESM file, ``bundle.min.mjs``, and embeds the manifest (the SDK version, the
+supervisor schema version, the entry file and, for each Dag declared in TypeScript, the file that declares it)
+after a leading compact JSON ``//# airflowBundle=...`` layout header. The layout records the byte ranges and
+SHA-256 digests of the manifest and executable code, so there is one file to deploy, with no separate manifest or
+``node_modules``.
 
 The code is minified because an integrity digest is only worth taking over an artifact nobody is expected to
 read or edit in place. Function names are kept through minification, since a task id defaults to its
@@ -517,9 +518,8 @@ that only supply utilities or types are not embedded.
     npm install --save-dev esbuild
     npx airflow-ts-pack src/main.ts --outdir dist
 
-Use ``--outdir <dir>`` to choose the output directory (default ``dist``), ``--outfile <path>`` to name the
-artifact exactly, which helps when one Dag bundle holds several bundles, and ``--source <name>`` to set
-the source name displayed in the Airflow UI (default: the entry file's basename). ``--outdir`` and
+Use ``--outdir <dir>`` to choose the output directory (default ``dist``), or ``--outfile <path>`` to name the
+artifact exactly, which helps when one Dag bundle holds several bundles. ``--outdir`` and
 ``--outfile`` are mutually exclusive, and an ``--outfile`` name must end in ``.min.mjs`` so the coordinator
 can find it.
 

airflow-core/tests/unit/dag_processing/test_task_handler_processor_go.py

@@ -25,8 +25,10 @@
 from typing import TYPE_CHECKING
 from unittest import mock
 
+import jsonschema
 import pytest
 import structlog
+import yaml
 
 from airflow.dag_processing.processor import TaskHandlerDeclaration, TaskHandlerParam
 from airflow.dag_processing.task_handler_processor import LangSDKTaskHandlerProcessorProcess
@@ -103,6 +105,28 @@ def _go_coordinator(monkeypatch, tmp_path):
         reset_coordinator_manager()
 
 
+def test_a_packed_go_bundle_records_no_dag_inventory(go_bundle):
+    completed = subprocess.run(
+        ["go", "tool", "airflow-go-pack", "inspect", os.fspath(go_bundle)],
+        cwd=GO_SDK_PATH,
+        env={**os.environ, "CGO_ENABLED": "0"},
+        capture_output=True,
+        text=True,
+        check=False,
+    )
+    assert completed.returncode == 0, completed.stderr
+
+    manifest = yaml.safe_load(completed.stdout)
+
+    assert manifest["sdk"]["language"] == "go"
+    assert manifest["sdk"]["supervisor_schema_version"]
+    assert "dags" not in manifest
+    schema = json.loads(
+        (AIRFLOW_ROOT_PATH / "task-sdk" / "docs" / "airflow-metadata.schema.json").read_text()
+    )
+    jsonschema.Draft202012Validator(schema).validate(manifest)
+
+
 def test_probes_the_task_handlers_of_a_packed_go_bundle(go_bundle):
     result = LangSDKTaskHandlerProcessorProcess.run(
         coordinator="go",

airflow-core/tests/unit/dag_processing/test_task_handler_processor_ts.py

@@ -114,6 +114,18 @@ def _declare(task_id: str) -> TaskHandlerDeclaration:
     return TaskHandlerDeclaration(task_id=task_id, binding="named", params=None)
 
 
+def test_a_packed_bundle_embeds_no_task_handlers(example_bundle):
+    metadata_line = example_bundle.read_bytes().split(b"\n")[1]
+    prefix = b"//# airflowMetadata="
+    assert metadata_line.startswith(prefix)
+
+    metadata = json.loads(metadata_line[len(prefix) :])
+
+    assert metadata["sdk"]["language"] == "typescript"
+    assert "dag_source_paths" in metadata
+    assert "task_handlers" not in metadata
+
+
 @pytest.mark.usefixtures("fresh_coordinator_manager")
 @conf_vars({("sdk", "coordinators"): json.dumps(COORDINATORS)})
 @mock.patch.object(supervisor, "_should_use_exec", autospec=True, return_value=False)

airflow-e2e-tests/tests/airflow_e2e_tests/conftest.py

@@ -543,9 +543,10 @@ def _setup_java_sdk_integration(dot_env_file, tmp_dir):
 def _run_go_sdk_pack(output_path, *, capture_output=False, native=False):
     """Run ``go tool airflow-go-pack`` natively or inside the pinned Go toolchain container.
 
-    ``go tool airflow-go-pack`` builds the bundle package, reads its
-    --airflow-metadata, and appends the source + airflow-metadata.yaml + the
-    AFBNDL01 trailer, writing a single self-contained executable bundle.
+    ``go tool airflow-go-pack`` builds the bundle package, reads the go-sdk
+    version from the binary's build information (it never runs the binary), and
+    appends the source + airflow-metadata.yaml + the AFBNDL01 trailer, writing a
+    single self-contained executable bundle.
     CGO_ENABLED=0 yields a fully static binary that runs on the stock worker.
 
     In ``native`` mode (used in CI, where the host already has a Go toolchain plus
@@ -559,10 +560,6 @@ def _run_go_sdk_pack(output_path, *, capture_output=False, native=False):
     * HOME points at a writable, gitignored dir under go-sdk/bin so the Go build
       and module caches persist between runs (first run downloads modules once;
       subsequent runs skip straight to compilation).
-    * USER/HOME must be set because the SDK calls user.Current() at init; with
-      cgo disabled Go's pure-Go resolver reads those env vars instead of libc,
-      and panics if either is empty (the same vars are set on the worker and
-      the Dag processor in go.yml so the packed binary runs the same way there).
     """
     if native:
         cwd = GO_SDK_ROOT_PATH

contributing-docs/30_new_language_sdk.rst

@@ -228,7 +228,10 @@ If the target runtime compiles to a self-contained native executable, the
 :class:`~airflow.sdk.coordinators.executable.ExecutableCoordinator` can
 discover and launch it automatically. For the coordinator to understand the
 bundle correctly, extra metadata should be appended to the executable by a
-custom bundling step at build-time.
+custom bundling step at build-time. The metadata names the SDK and the
+supervisor schema version and lists no Dags or tasks: the Dag processor asks the
+bundle which task handlers it registers, so the bundling step does not need to
+run the executable.
 
 See :ref:`Executable Bundle Spec` in Task SDK documentation for details.
 
@@ -536,9 +539,10 @@ authored in the target language. An SDK declares each one independently.
     The task can read, write, and delete Airflow Variables.
 
 ``self-contained-bundle`` (MUST)
-    The SDK's build artifact embeds its own Airflow metadata (``dag_id``, ``task_id`` and
-    the rest of the task descriptor) inside the *same* artifact as the task code, rather
-    than shipping it in a separate sidecar file, so the deployable unit is self-describing.
+    The SDK's build artifact embeds its own Airflow metadata (what Airflow needs to run it,
+    such as the supervisor schema version it was built against) inside the *same* artifact
+    as the task code, rather than shipping it in a separate sidecar file, so the deployable
+    unit is self-describing.
     Each runtime satisfies this its own way — a Go binary carries an ``AFBNDL01`` metadata
     trailer (see `Native Executable Bundle Format`_), a JVM artifact embeds it in the jar,
     a Node bundle embeds it in the package.

go-sdk/README.md

@@ -37,10 +37,10 @@ another.
 Python tasks are imported and run in-process. Go is compiled, so the model is different.
 
 A single binary that bundles one or more Dags' task functions is called a **bundle**. You build one with
-the SDK's packer, `airflow-go-pack`, which compiles your code and appends a metadata footer (the manifest
-of `dag_id`s and `task_id`s, plus the Dag source) to the executable. The result is a **self-contained
-executable bundle**: a single runnable file that *is* the bundle, with no separate manifest or archive to
-ship alongside it.
+the SDK's packer, `airflow-go-pack`, which compiles your code and appends a metadata footer (a small
+manifest with the SDK and supervisor schema versions, plus the Dag source) to the executable. The result
+is a **self-contained executable bundle**: a single runnable file that *is* the bundle, with no separate
+manifest or archive to ship alongside it.
 
 ## You still need a Python stub Dag (for now)
 
@@ -265,17 +265,14 @@ the full range of task states, and alternate XCom backends without implementing
     ./example/bundle
   ```
 
-  Alternatively, use `--executable`/`--source`. The packer normally execs the binary to read
-  its metadata; a cross-compiled binary cannot run on the host, so generate the metadata on a machine that
-  can run it and pass the file with `--airflow-metadata`:
+  Alternatively, use `--executable`/`--source`. The packer never runs the binary, so one built for any
+  platform packs on any host. It reads the go-sdk version from the binary's build information and refuses a
+  binary built against a different go-sdk version than the packer's, so run it from the module that builds
+  the bundle:
 
   ```bash
-  # on linux/amd64 machine:
-  go build -o my-bundle ./example/bundle
-  ./my-bundle --airflow-metadata > airflow-metadata.yaml
-
-  # on darwin/arm64 machine:
-  go tool airflow-go-pack --executable ./my-bundle --source main.go --airflow-metadata airflow-metadata.yaml
+  GOOS=linux GOARCH=amd64 go build -o my-bundle ./example/bundle
+  go tool airflow-go-pack --executable ./my-bundle --source ./example/bundle/main.go
   ```
 
   > [!NOTE]

go-sdk/adr/0001-bundle-packing-options.md

@@ -44,6 +44,11 @@ options below still describe valid *packer mechanisms*; only the
 artefact each one writes has changed from a ZIP to a footer-augmented
 executable.
 
+Option D is retired: the bundle binary no longer accepts `--airflow-metadata`,
+and the packer no longer runs the binary but reads the go-sdk version from
+its build information (see
+[ADR-0014](../../airflow-core/adr/lang-sdk/0014-bundle-metadata-and-cache-digest.md)).
+
 ## Context
 
 The executable provider's bundle spec

go-sdk/adr/0002-use-go-tool-directive-for-bundle-packer.md

@@ -36,6 +36,12 @@ A standalone binary + Option D introspection contract + Option H
 ZIP output; read them with the ADR 0004 substitution in mind, and
 treat ADR 0004 as authoritative wherever the two disagree.
 
+The `--airflow-metadata` introspection this ADR has the packer run is retired:
+the packer no longer runs the binary but reads the go-sdk version from its
+build information, and neither the bundle binary nor the packer accepts
+`--airflow-metadata` any more (see
+[ADR-0014](../../airflow-core/adr/lang-sdk/0014-bundle-metadata-and-cache-digest.md)).
+
 ## Context
 
 [ADR 0001](0001-bundle-packing-options.md) enumerated nine candidate

go-sdk/adr/0003-coordinator-protocol-msgpack-ipc.md

@@ -40,6 +40,10 @@ decision in this ADR is unaffected: the binary still honours
 the container format it ships inside. Read the ZIP mentions below with
 the ADR 0004 substitution in mind.
 
+The `--airflow-metadata` mode described below is removed, so the bundle binary
+speaks only the coordinator protocol (see
+[ADR-0014](../../airflow-core/adr/lang-sdk/0014-bundle-metadata-and-cache-digest.md)).
+
 ## Context
 
 A Go SDK bundle binary today (the artefact built from

go-sdk/adr/0004-self-contained-executable-bundle.md

@@ -30,6 +30,11 @@ packer mechanism (Option A standalone packer + Option D introspection
 contract + Option H `tool` directive) is unchanged; only the artefact
 the packer writes is changed.
 
+The `--airflow-metadata` introspection this ADR has the packer run is retired:
+the packer no longer runs the binary but reads the go-sdk version from its
+build information, and the manifest no longer lists Dags (see
+[ADR-0014](../../airflow-core/adr/lang-sdk/0014-bundle-metadata-and-cache-digest.md)).
+
 ## Context
 
 ADR 0001 / ADR 0002 picked a ZIP archive as the bundle container,

go-sdk/adr/0005-retire-go-edge-worker.md

@@ -26,6 +26,10 @@ Date: 2026-08-20
 Accepted. Supersedes the dual-runtime portion of
 [ADR 0003](0003-coordinator-protocol-msgpack-ipc.md).
 
+`--airflow-metadata` has since been removed, so it is no longer available to the
+bundle packer (see
+[ADR-0014](../../airflow-core/adr/lang-sdk/0014-bundle-metadata-and-cache-digest.md)).
+
 ## Context
 
 The Go SDK supported two task-execution architectures. The standalone Go Edge

go-sdk/airflow/bundle.go

@@ -125,8 +125,8 @@ func (b *BundleRef) Register(items ...Registerable) {
 }
 
 // taskHandlerMap holds the registered task handlers by dag_id and task_id.
-// It also keeps registration order. The --airflow-metadata manifest, and the reply to the Dag
-// processor's task handler parse, list the tasks of each Dag in that order.
+// It also keeps registration order. The reply to the Dag processor's task handler parse lists
+// the tasks of each Dag in that order.
 type taskHandlerMap struct {
 	mu       sync.RWMutex
 	handlers map[string]map[string]bundle.Task