Skip to content

syncables 0.18.0: browser-safe read path as syncables/browser - #83

Merged
michielbdejong merged 1 commit into
mainfrom
claude/syncables-browser-entry
Sep 23, 2026
Merged

michielbdejong merged 1 commit into
mainfrom
claude/syncables-browser-entry

Conversation

@michielbdejong

Copy link
Copy Markdown
Contributor

Merging this publishes syncables@0.18.0 to npm. This PR bumps syncables/package.json from 0.17.1 to 0.18.0. When that change reaches main, .github/workflows/syncables-publish.yml runs npm publish through npm Trusted Publishing. The publish only succeeds once the npm Trusted Publisher for syncables points at ontola/atomic-plugins and syncables-publish.yml.

Why

Browser and iframe plugins, starting with pets and Notion, need syncables' read path. The only entry today is syncables, and it imports node:http, node:fs and node:crypto, so it can't be bundled for a browser. The closed PR #76 got around this with a 1,100-line port, integrations/localthought/reflector-read.ts. This PR moves that read path into syncables itself.

What

  • New entry point syncables/browser (src/browser.ts, "exports": { "./browser": … }). It contains the read path only and never calls fetch. Every request goes through an injected Transport, a function from { url, method, headers, body? } to { status, headers, body }. fetchTransport(fetch) adapts anything fetch-shaped. The entry exports:
    • prepareDocument(document, overlays): applies overlays, then resolves $refs.
    • describePlatform(document): returns { parameters, collections, upstream }, read from components.crudResources.
    • readPlatform(document, { platform, constants, transport, selection?, limits?, probe? }): returns { platform, ontology, records, errors }. This is the same shape as FetchedPlatform in integrations/localthought.
    • paginate(document, { transport, path, method, … }): walks one operation's pages and doesn't need crudResources.
    • Also deriveOntology, ontologyShortname, mergeQuerySelections, DATATYPES, DEFAULT_READ_LIMITS, and the existing pagination and overlay helpers.
  • Request-body pagination. buildBody builds a POST's JSON body from request.bodyFields: cursor, page size, offset and page. nextCursor now reads body-field offset and page roles. On a crudResources collection, x-list-method: POST lists with a POST instead of a GET, and x-list-body adds fixed body fields. This covers Notion's /v1/search and /v1/databases/{id}/query, which send start_cursor in the request and return next_cursor. Auto-detection already matched body fields, so an operation that declares start_cursor and page_size in its request body picks up this scheme without x-pagination.
  • Node-only code stays separate. applyOverlay moves to openapi/apply-overlay.ts, which has no file-system access. openapi/overlay.ts re-exports it and keeps loadOverlay. The mock server and loaders are unchanged.
  • The Node API is unchanged. syncables also exports the read path, with paginate renamed paginateOperation so it doesn't clash with the PaginateOptions type that ApiClient.paginate uses.

Limits and what isn't covered

  • A read defaults to at most 10,000 requests, 5,000 records and 30 minutes. A 429 is retried at most 3 times per request, after its Retry-After. When any of these limits is reached, the read stops, keeps what it has read so far, and reports the stop in errors.
  • A nextLink to another origin, or with credentials or a fragment, stops that collection with an error. So does a page that repeats, which is what happens when a proxy drops the cursor.
  • The browser entry doesn't include js-yaml, so pass documents and overlays as parsed JSON.
  • Not ported from PR Move LocalThought setup/sync onto the reflector read path (PlatformReader) #76: standalone x-crud reads and cross-collection links. Header-based auto-detection (matchHeaders) is still not implemented. Neither of these was in the Node API before.
  • Error messages say "Read …" where PR Move LocalThought setup/sync onto the reflector read path (PlatformReader) #76 said "Import …". For example, Read incomplete: … replaces Import incomplete; no changes proposed: ….

Tests

  • __tests__/unit/browser/bundle.test.ts bundles src/browser.ts with esbuild platform: 'browser'. It fails if anything imports a Node built-in, pulls in a node_modules file, or uses process., Buffer. or require(. A control test bundles the Node entry and checks that it is flagged, so the check isn't vacuous.
  • __tests__/unit/read/read.test.ts (18 tests, all with a stub transport) covers:
  • request-builder.test.ts adds buildBody cases.

Verification

All run locally on Node 22.23.2:

  • The steps in syncables-ci.yml: npm ci, npm run lint, npm run prettier:check, npm test (127 tests pass) and npm run build.
  • npm pack --dry-run shows syncables@0.18.0 with 53 files, including build/src/browser.{js,d.ts} and build/src/read/*.
  • I installed the packed tarball into a scratch consumer and imported syncables/browser:
    • tsc resolved its types with moduleResolution: bundler.
    • esbuild platform: 'browser' bundled it, and the bundle contains no node: imports.
    • The bundle ran and read a stubbed collection.
  • Bundling the release build with esbuild platform: 'browser' and --minify gives 19.2 kB.

package-lock.json and pnpm-lock.yaml each gain only an esbuild ~0.21.5 devDependency. That is the version vitest already installs.

🤖 Generated with Claude Code

Browser and iframe plugins (pets, Notion) need syncables' read path, but
the package entry imports node:http, node:fs and node:crypto, so it can't be
bundled for a browser. The new syncables/browser entry exports the read
path alone, with an injected Transport.

- src/read/: crudResources collection discovery, query selections, a page
  walker with request and record budgets and 429 Retry-After handling,
  ontology derivation with Atomic Data datatypes, readPlatform and paginate.
  Re-derived from the browser port in closed PR #76
  (integrations/localthought/reflector-read.ts). The page walker reuses
  syncables' own pagination modules.
- Request-body pagination: buildBody puts cursor, page-size, offset and
  page fields from request.bodyFields into a POST's JSON body, and
  nextCursor now reads body-field offset and page roles. Collections opt in
  to POST listing with x-list-method: POST and add fixed body fields with
  x-list-body. This covers Notion's /v1/search and
  /v1/databases/{id}/query.
- applyOverlay moves to openapi/apply-overlay.ts, which has no file-system
  access. openapi/overlay.ts re-exports it and keeps loadOverlay, so the
  Node API is unchanged. The main entry also exports the read path, with
  paginate renamed paginateOperation.
- Tests: an esbuild platform: 'browser' bundle of src/browser.ts that
  fails on any Node built-in import, with a control test showing that the
  Node entry is caught. Also read-path unit tests with a stub transport:
  the PR #76 pets, calendar and workspace scenarios, and a Notion-shaped
  POST-body scenario built from overlays.

The session log is docs/ai-logs/sessions/2026-09-23-browser-entry.md. The
session URL was not available to the agent, so this commit has no
Claude-Session trailer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant