Skip to content

fix(sandbox): move SpritesClient to apps/web IO boundary (fixes ERR_REQUIRE_ESM) - #1667

Merged
2witstudios merged 3 commits into
masterfrom
fix/sprites-esm-io-boundary
Jun 21, 2026
Merged

2witstudios merged 3 commits into
masterfrom
fix/sprites-esm-io-boundary

Conversation

@2witstudios

@2witstudios 2witstudios commented Jun 21, 2026 •

Copy link
Copy Markdown
Owner

Problem

Terminal commands and agent bash/writeFile/readFile tools were silently failing — Sprites API saw zero traffic.

Root cause: @pagespace/lib compiles to CommonJS (module: commonjs in tsconfig.build.json). The static import { SpritesClient } from '@fly/sprites' in sprites.ts compiled to require('@fly/sprites') in the dist. Node.js rejects require() of an ESM-only package with ERR_REQUIRE_ESM. The route's outer try/catch caught it and returned 500 before Sprites was ever called.

The previous fix (#1665 — serverExternalPackages: ["@fly/sprites"]) changed the error from "Cannot find package" to ERR_REQUIRE_ESM but didn't fix the underlying cause. The Next.js docs confirm serverExternalPackages uses native Node.js require() at runtime — wrong for an ESM-only package.

Fix: pure-core / IO-shell split

@pagespace/lib is the pure core — it should hold interfaces and transformation logic only, no IO side effects from its own imports. Instantiating SpritesClient is an IO effect and belongs at the application boundary (apps/web), where Next.js bundles it as an ESM-aware server-side webpack chunk instead of emitting require().

Changes

  • packages/lib/.../sprites.ts — import { SpritesClient } → import type { NetworkPolicy } (type-only, erased at compile time). Remove defaultSdk(). Make sdk required: createSpritesSandboxClient({ sdk: SpritesSdk }).
  • apps/web/src/lib/sandbox/sprites-client.ts (new) — the only file that imports @fly/sprites. Lazily creates and caches the SpritesClient, returns an ExecSandboxClient via the lib factory. Next.js/webpack bundles this as a server async chunk (ESM → CJS-compatible). @fly/sprites must NOT be in serverExternalPackages — that forces require() which fails for ESM-only packages.
  • apps/web/package.json — add @fly/sprites as a direct dep so webpack can resolve and bundle it and TypeScript can find its types.
  • next.config.ts — remove @fly/sprites from serverExternalPackages (wrong fix: forces require() at runtime). Add a comment explaining why it must not be listed there.
  • terminal/execute/route.ts and sandbox-tools-runtime.ts — import createProductionSpritesSandboxClient from the new app-boundary module instead of from @pagespace/lib.
  • Fix stale "Node 22" comment in sandbox-tools-runtime.ts (web image has been 24.x).

Tests

  • All 23 sprites unit tests pass. Tests inject a fake sdk and are unaffected by the interface change.
  • Updated terminal/execute/route.test.ts to mock @/lib/sandbox/sprites-client (the new import path) and stub createProductionSpritesSandboxClient as mockResolvedValue (async, matching the route's await call). All 21 terminal route tests pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_01X2ogJhXAnbtKnbGikjZz31

@pagespace/lib compiles to CommonJS (tsconfig.build.json: module=commonjs),
so the static `import { SpritesClient } from '@fly/sprites'` in sprites.ts
was compiled to `require('@fly/sprites')` in the dist. Node.js rejects
CJS require() of an ESM-only package with ERR_REQUIRE_ESM, causing every
terminal command and every agent bash/writeFile/readFile tool call to fail
silently (caught by the route's outer try/catch, returned as 500) — no
traffic ever reached the Sprites API.

The previous fix (adding @fly/sprites to serverExternalPackages) put the
package in standalone node_modules but the docs confirm serverExternalPackages
uses native Node.js require() at runtime, which fails identically.

Fix: pure-core / IO-shell split.

- Remove the static SpritesClient import from @pagespace/lib (type-only
  import type { NetworkPolicy } replaces it; erased at compile time).
- Remove defaultSdk() from lib; make sdk required in
  createSpritesSandboxClient({ sdk }) so the call site must provide it.
- Add apps/web/src/lib/sandbox/sprites-client.ts: the only file that
  imports @fly/sprites. Lives in apps/web where Next.js bundles it as
  a server-side async webpack chunk (ESM → CJS-compatible bundle) rather
  than emitting require(). SpritesClient is lazily instantiated and
  cached for the process lifetime.
- Add @fly/sprites as a direct dependency of apps/web so webpack can
  resolve and bundle it, and TypeScript can find its types.
- Remove @fly/sprites from serverExternalPackages (wrong fix: forces
  require() which fails for ESM-only packages; webpack bundling is correct).
- Update both call sites (terminal route, sandbox-tools-runtime) to import
  createProductionSpritesSandboxClient from the new app-boundary module.
- Fix stale Node 22 comment in sandbox-tools-runtime (web image is 24.x).

All 23 sprites unit tests pass. Tests inject a fake sdk and are unaffected
by the interface change.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2ogJhXAnbtKnbGikjZz31
@coderabbitai

coderabbitai Bot commented Jun 21, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The @fly/sprites SDK instantiation is moved from packages/lib into a new apps/web/src/lib/sandbox/sprites-client.ts module that lazily imports and caches the SDK via a dynamic import(). createSpritesSandboxClient in packages/lib is updated to require an injected sdk argument. Both call sites in the terminal execute route and sandbox-tools-runtime switch to the new createProductionSpritesSandboxClient factory. @fly/sprites is removed from serverExternalPackages and added as a direct apps/web dependency.

Changes

Fly Sprites ESM Interop Refactor

Layer / File(s) Summary
sprites.ts: remove default SDK, require injection
packages/lib/src/services/sandbox/sandbox-client/sprites.ts
Removes the direct @fly/sprites runtime import and the internal defaultSdk fallback. createSpritesSandboxClient now requires a caller-supplied sdk: SpritesSdk argument with no default.
New lazy sprites-client factory
apps/web/src/lib/sandbox/sprites-client.ts
Adds a new module that dynamically imports @fly/sprites on first call, caches the SpritesSdk instance for the process lifetime, and exports createProductionSpritesSandboxClient() which passes the SDK into createSpritesSandboxClient.
Call-site wiring, Next.js config, and package dep
apps/web/src/app/api/pages/[pageId]/terminal/execute/route.ts, apps/web/src/lib/ai/tools/sandbox-tools-runtime.ts, apps/web/next.config.ts, apps/web/package.json
Both call sites switch to createProductionSpritesSandboxClient from @/lib/sandbox/sprites-client. @fly/sprites is removed from serverExternalPackages so Next.js bundles it, and added as a direct apps/web dependency at 0.0.1-rc37.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

Possibly related PRs

  • 2witstudios/PageSpace#1664: Both PRs modify the POST /terminal/execute route's sandbox client acquisition path — this PR changes how the sprites client is created, while #1664 adds authorization checks in the same handler.

Poem

🐇 Hop, hop — no more ESM fright,
The sprites are loaded lazily at night.
A dynamic import, cached just right,
The lib stays pure, the app takes flight.
No more module graph woes to fight!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The PR title clearly and specifically summarizes the main architectural change: moving SpritesClient instantiation to the app/IO boundary to fix an ESM/CommonJS compatibility error.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/sprites-esm-io-boundary

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 004207f2e9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

const [store, { createProductionSpritesSandboxClient }] = await Promise.all([
createDbTerminalSessionStore(),
import('@pagespace/lib/services/sandbox/sandbox-client/sprites'),
import('@/lib/sandbox/sprites-client'),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Mock the new Sprites factory in terminal tests

The terminal route tests still mock/import @pagespace/lib/services/sandbox/sandbox-client/sprites (checked apps/web/src/app/api/pages/[pageId]/terminal/execute/__tests__/route.test.ts), so after this import moved to @/lib/sandbox/sprites-client, the post-slot tests instantiate the real SpritesClient and happy-path cases call the real Sprites API instead of the fake client. Update the tests to mock the new app-boundary module, otherwise the web test suite becomes dependent on real Sprites credentials/network and fails or flakes.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commits 60dd74b8b and 34a8d1a9a:

  • fix(terminal-tests): mock new sprites-client app boundary instead of lib path — updated route.test.ts to mock @/lib/sandbox/sprites-client instead of the old @pagespace/lib/... path, and changed from mockReturnValue (sync) to mockResolvedValue (async) to match the new await createProductionSpritesSandboxClient() call in the route. All 21 tests in the file now pass.
  • fix(sprites-client): correct comments — webpack bundles, not serverExternalPackages — updated both comment blocks in sprites-client.ts to accurately describe the mechanism: webpack bundles @fly/sprites as a server async chunk because it is a direct dep of apps/web (not in serverExternalPackages).

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
apps/web/src/lib/sandbox/sprites-client.ts (1)

23-39: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick win

Memoize in-flight SDK initialization to avoid duplicate client creation under concurrency.

Between Line 26 and Line 38, simultaneous first calls can race and create multiple SpritesClient instances. Memoize the initialization promise in addition to cachedSdk.

Suggested refactor
 let cachedSdk: SpritesSdk | null = null;
+let cachedSdkPromise: Promise<SpritesSdk> | null = null;
 
 async function getSpritesSDK(): Promise<SpritesSdk> {
   if (cachedSdk) return cachedSdk;
+  if (cachedSdkPromise) return cachedSdkPromise;
-  // Dynamic import: `@fly/sprites` is in serverExternalPackages so webpack emits
-  // a native import() here rather than require() — the only form that works for
-  // an ESM-only package from a CJS runtime.
-  const { SpritesClient } = await import('`@fly/sprites`');
-  const client = new SpritesClient(resolveSpritesToken());
-  cachedSdk = {
-    getSprite: (name) => client.getSprite(name) as unknown as Promise<SpriteInstanceLike>,
-    createSprite: (name, config) =>
-      client.createSprite(name, config) as unknown as Promise<SpriteInstanceLike>,
-    deleteSprite: (name) => client.deleteSprite(name),
-  };
-  return cachedSdk;
+  cachedSdkPromise = (async () => {
+    const { SpritesClient } = await import('`@fly/sprites`');
+    const client = new SpritesClient(resolveSpritesToken());
+    cachedSdk = {
+      getSprite: (name) => client.getSprite(name) as unknown as Promise<SpriteInstanceLike>,
+      createSprite: (name, config) =>
+        client.createSprite(name, config) as unknown as Promise<SpriteInstanceLike>,
+      deleteSprite: (name) => client.deleteSprite(name),
+    };
+    return cachedSdk;
+  })().finally(() => {
+    cachedSdkPromise = null;
+  });
+  return cachedSdkPromise;
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/web/src/lib/sandbox/sprites-client.ts` around lines 23 - 39, The
getSpritesSDK function has a race condition where concurrent calls can bypass
the cachedSdk check and create multiple SpritesClient instances simultaneously.
In addition to the cachedSdk variable, introduce another variable to memoize the
in-flight initialization promise. Check if initialization is already in progress
and return that promise, otherwise start a new initialization only when both
cachedSdk is null and no in-flight promise exists. This ensures that concurrent
callers will await the same initialization promise rather than creating
duplicate clients.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/web/src/lib/sandbox/sprites-client.ts`:
- Around line 4-9: The comments in the sprites-client.ts file contain
contradictory guidance about the serverExternalPackages configuration. Update
the comment block starting at line 4 (which discusses `@fly/sprites` being
ESM-only and mentions serverExternalPackages) and the comment block at line
27-29 to correctly reflect that `@fly/sprites` should NOT be included in
serverExternalPackages, matching the actual configuration in the next.config.ts
file. Remove any references that suggest `@fly/sprites` should be added to
serverExternalPackages and clarify the current runtime model that handles the
ESM import correctly without it.

---

Nitpick comments:
In `@apps/web/src/lib/sandbox/sprites-client.ts`:
- Around line 23-39: The getSpritesSDK function has a race condition where
concurrent calls can bypass the cachedSdk check and create multiple
SpritesClient instances simultaneously. In addition to the cachedSdk variable,
introduce another variable to memoize the in-flight initialization promise.
Check if initialization is already in progress and return that promise,
otherwise start a new initialization only when both cachedSdk is null and no
in-flight promise exists. This ensures that concurrent callers will await the
same initialization promise rather than creating duplicate clients.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 54437d03-872d-4f11-8585-d955fe200191

📥 Commits

Reviewing files that changed from the base of the PR and between c3d79e6 and 004207f.

⛔ Files ignored due to path filters (1)
  • bun.lock is excluded by !**/*.lock
📒 Files selected for processing (6)
  • apps/web/next.config.ts
  • apps/web/package.json
  • apps/web/src/app/api/pages/[pageId]/terminal/execute/route.ts
  • apps/web/src/lib/ai/tools/sandbox-tools-runtime.ts
  • apps/web/src/lib/sandbox/sprites-client.ts
  • packages/lib/src/services/sandbox/sandbox-client/sprites.ts

Comment thread apps/web/src/lib/sandbox/sprites-client.ts Outdated
2witstudios and others added 2 commits June 21, 2026 16:06
…ternalPackages

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2ogJhXAnbtKnbGikjZz31
…lib path

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X2ogJhXAnbtKnbGikjZz31
@2witstudios
2witstudios merged commit e332719 into master Jun 21, 2026
10 checks passed
@2witstudios
2witstudios deleted the fix/sprites-esm-io-boundary branch August 14, 2026 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant