Skip to content

Pin down that external vocabulary served as octet-stream parses - #1845

Closed
michielbdejong wants to merge 1 commit into
claude/vocab-proxy-signingfrom
claude/vocab-octet-stream
Closed

michielbdejong wants to merge 1 commit into
claude/vocab-proxy-signingfrom
claude/vocab-octet-stream

Conversation

@michielbdejong

Copy link
Copy Markdown
Contributor

Refs #1837 #1838

Stacked on #1841.

GitHub Pages serves the shared ontology (https://ontola.github.io/atomic-plugins/ontology/...) as application/octet-stream, because the files have no extension. This PR checks every path that fetches external resources for that case.

No code change was needed. None of the three paths looks at the Content-Type. Each one goes by the body:

Path Where What decides
Browser client, direct Client.fetchResourceHTTP (browser/lib/src/client.ts) status 200, then JSON.parse + the JSON-AD parser
/path proxy fallback (#1841) server/src/plugins/path.rs → get_path → fetch_resource same as the server fetch below; the proxy answers application/ad+json itself
Server's own fetch fetch_resource / fetch_body_with_client (lib/src/client/helpers.rs) status 200, then parse_json_ad_string

So JSON-AD parses under any type, and HTML or binary data is refused under any type because it isn't JSON. This PR only adds tests, so that a future Content-Type check can't silently break the shared ontology.

Tests

  • browser/lib/src/vocab-proxy-fallback.test.ts: the stubbed vocabulary host now answers application/octet-stream like Pages; the proxy answers application/ad+json. New cases check that the client accepts JSON-AD served as octet-stream, as text/plain and with no Content-Type, and refuses HTML (as octet-stream and as text/html) and binary data served as octet-stream.
  • browser/lib/tests/vocab-proxy.integration.test.ts (real server): the stub already served octet-stream. Added a direct fetch from it, and HTML served as octet-stream refused both directly and through /path.
  • lib/src/client/helpers.rs content_type_tests: fetch_resource against a loopback server, where JSON-AD as octet-stream is accepted and HTML or PNG bytes as octet-stream are refused.

Mutation check. Since there was nothing to fix, "fails before" was shown by temporarily adding a strict content-type contains json check:

  • In the TS client, it failed the four octet-stream, text/plain and no-type unit cases and the direct integration case.
  • In Rust fetch_body_with_client, it failed the Rust accept test and the proxied and fallback integration cases.

All passed again after the check was removed.

Runs:

  • browser/lib vitest: 1040 passed.
  • Integration test: 5 passed.
  • cargo test -p atomic_lib --features db-redb --lib: 655 passed. The one failure, startup_compaction_shrinks_a_bloated_store_and_keeps_every_resource, is flaky under load and passes alone.
  • cargo test -p atomic-server --lib: 270 passed.
  • Clippy -D warnings, oxlint/oxfmt and tsc are clean.

Live check

https://ontola.github.io/atomic-plugins/ontology/v1 is already live:

  • It returns 200, content-type: application/octet-stream, access-control-allow-origin: *, and a preflight gets 405.
  • classes/event-v1 is also 200 octet-stream.
  • A one-off run against a local server fetched ontology/v1 and classes/event-v1 through @tomic/lib's Client, both directly and through the server's signed /path. All four succeeded with the right subject.

🤖 Generated with Claude Code

GitHub Pages serves the shared ontology's extensionless JSON-AD files as
`application/octet-stream`. None of the external fetch paths looks at the
Content-Type: the browser client, the server behind `/path`, and the
server's own `fetch_resource` all go by the body. JSON-AD parses under any
type, and HTML or binary is refused under any type because it is not JSON.

These tests keep it that way, since a Content-Type check would break the
shared ontology silently:

- Unit stubs serve terms as `application/octet-stream`, like Pages, and the
  client is checked on octet-stream, text/plain and a missing type, and
  refuses HTML and binary served as octet-stream.
- The integration test adds a direct fetch from the octet-stream stub, and
  HTML under that type is refused both directly and through the proxy.
- A Rust test covers `fetch_resource` on a loopback server for the same
  three cases.

A strict `json` Content-Type check, added temporarily, failed each of them.

Refs #1837 #1838

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
joepio added a commit that referenced this pull request Sep 26, 2026
…1845) (#1854)

Combined batch, green on Mancave in run 4658 on 0a0b9ce (CI and Downstream).

Includes #1847 and #1845's own commit. #1842 was taken out and waits for a fix to its new test.

joepio commented Sep 26, 2026

Copy link
Copy Markdown
Member

This PR's own commit (fa72d4c) landed on develop in 2a17979 via the combined batch #1854, so I'm closing it. The other commits on this branch belong to #1842, which stays open.


Generated by Claude Code

@joepio joepio closed this Sep 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants