Skip to content

Lazy-load the Code tab grammar from the Dag's stored language - #74643

Merged
kaxil merged 1 commit into
apache:mainfrom
astronomer:lazy-load-dag-source-grammars
Oct 13, 2026
Merged

kaxil merged 1 commit into
apache:mainfrom
astronomer:lazy-load-dag-source-grammars

Conversation

@pierrejeambrun

@pierrejeambrun pierrejeambrun commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Follow-up to #74633.

Dag importers are pluggable: the YAML importer (#74586), the Go/Java/TypeScript coordinators, and any third-party importer each store their own language id in dag_code.language. Core can't know that list, so hardcoding grammars in the UI means a custom importer (say a json or rust one) gets plain-text Dag source in the Code tab until someone patches the UI.

Monaco already ships a lazy registration for every Monarch grammar it bundles. Registering that instead lets the Code tab highlight whatever language the API returns, fetching a grammar chunk only the first time a Dag source uses it. The UI stops listing grammars and no longer loads the Go/Java/TypeScript/YAML grammars up front for Python-only deployments. Python keeps the patched f-string grammar (#67986), now loaded lazily too.

Unknown ids (for example the "text" placeholder written when a source can't be read) still fall back to plain text.

Python grammar is always loaded when opening the code tab (no python at all is very unlikely). Opening a YAML Dag loads the YAML grammar chunk in addition. (Same for other sdk languages)

YAML Dag Python Dag
yaml python

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5.5)

Generated-by: Claude Code (Opus 5.5) following the guidelines

@boring-cyborg boring-cyborg Bot added the area:UI Related to UI/UX. For Frontend Developers. label Oct 13, 2026
Dag importers are pluggable, so core cannot list every language an importer may
store in dag_code.language. Registering monaco's lazy grammar loaders lets the
Code tab highlight any language monaco ships, fetching a grammar only when a
Dag source first uses it.

@ashb ashb left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So how does this work with the compiled assets? Do those monaco editr languages still get build and compiled/minified/fingerprinted etc?

@pierrejeambrun

pierrejeambrun commented Oct 13, 2026 •

Copy link
Copy Markdown
Member Author

So how does this work with the compiled assets? Do those monaco editr languages still get build and compiled/minified/fingerprinted etc?

Yes, they go through the normal Vite production build like everything else. They're compiled, minified and fingerprinted, and shipped in ui/dist inside the airflow-core
wheel. Nothing is fetched from a CDN at runtime.

What changes is how they're split into files:

  • A small registration file. register.all-.js is about 18 kB (about 4.5 kB gzipped). It lists every language's id, extensions and aliases plus a loader, and is
    loaded with the editor.
  • One file per grammar. Each grammar gets its own fingerprinted file, e.g. kotlin-BOotOW0E.js, yaml-CQesJ-Gr.js, python-DMN_ArqD.js. The loader in the registration file
    points at the hashed name: loader: () => import("./kotlin-BOotOW0E.js"). A browser only downloads a grammar the first time it shows a Dag in that language. Because the
    name changes whenever the content does, browsers can cache these files safely.

Before, only five grammar files (Python, TypeScript, Java, Go, YAML) were built, and the editor loaded all of them up front. Now all 89 grammar files are built: about 490
kB raw, about 100 kB gzipped. That's the cost of supporting any language a custom importer reports, but it only adds to the wheel and dist size. A user never downloads
more than the grammars for the Dags they actually open.

@kaxil
kaxil merged commit 518f8d2 into apache:main Oct 13, 2026
118 of 129 checks passed
@kaxil
kaxil deleted the lazy-load-dag-source-grammars branch October 13, 2026 20:56
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-4-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
❌ v3-4-test Commit Link

You can attempt to backport this manually by running:

cherry_picker 518f8d2 v3-4-test

This should apply the commit to the v3-4-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-4-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
✅ v3-4-test PR Link

vatsrahul1001 pushed a commit that referenced this pull request Oct 18, 2026
…uage (#74643) (#74987)

Dag importers are pluggable, so core cannot list every language an importer may
store in dag_code.language. Registering monaco's lazy grammar loaders lets the
Code tab highlight any language monaco ships, fetching a grammar only when a
Dag source first uses it.
(cherry picked from commit 518f8d2)

Co-authored-by: Pierre Jeambrun <pierrejbrun@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UI Related to UI/UX. For Frontend Developers.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants