Skip to content

feat: add client-side JSON Schema 2020-12 preservation scenario - #335

Merged
pcarleton merged 5 commits into
modelcontextprotocol:mainfrom
pugsatoshi:feat/101-client-schema-preservation
Jul 31, 2026
Merged

feat: add client-side JSON Schema 2020-12 preservation scenario#335
pcarleton merged 5 commits into
modelcontextprotocol:mainfrom
pugsatoshi:feat/101-client-schema-preservation

Conversation

@pugsatoshi

Copy link
Copy Markdown
Contributor

This PR adds a new JSON-Schema-2020-12-preservation client conformance scenario that closes Issue #101. The existing json-schema-2020-12 scenario tests server-side keyword emission; this new scenario tests the client-side preservation path that SEP-1613 and TypeScript SDK PR #1135 actually fixed (.catchall() so the parser retains the 2020-12 vocabulary).

The scenario sets up a mock MCP server (via the version-aware MockServer abstraction from #321) that advertises:

  • json_schema_2020_12_tool — focal tool whose inputSchema contains the full SEP-1613 + SEP-2106 vocabulary
  • json_schema_echo — permissive echo tool

The client under test calls tools/list, then round-trips the observed inputSchema back via tools/call json_schema_echo with { schema: <observed> }.
The scenario diffs the echoed schema against the canonical fixture and emits eight per-keyword checks. SEP-2106 vocabulary checks are soft-gated via sep2106KeywordCheckStatus so they only fail on DRAFT-2026-v1 runs.

Motivation and Context

Closes #101.

Issue #101 noted that the existing json-schema-2020-12 scenario covers only the server's wire output, while the actual SEP-1613 / TS SDK PR #1135 fix was on the client side.
This PR fills that gap with a dedicated client-side scenario that mirrors the server-side scenario, one check at a time.

Scope is intentionally pinned to introducedIn: '2025-11-25'.
The 2026-07-28-RC changelog (draft changelog) does not change the preservation requirements for SEP-1613 / SEP-2106, so the wire shape under test remains unchanged.
The broader 2026-07-28-RC changes (SEP-2575 stateless, SEP-2549 CacheableResult, SEP-2243 Mcp-Method MUST) are deliberately out of scope for this PR; once 2026-07-28 is finalized and lands in DATED_SPEC_VERSIONS, the same scenario can be re-targeted in a follow-up.

The focal-schema fixture (a 2020-12 inputSchema with $schema, $defs.address.$anchor, allOf/anyOf, if/then/else, additionalProperties: false) was previously embedded only in the server-side scenario's description markdown block.
This PR extracts it into an exported JSON_SCHEMA_2020_12_FIXTURE, so the new scenario can reuse it verbatim; the server-side description now derives its example from the same constant via JSON.stringify, eliminating the risk of documentation/runtime drift.

How Has This Been Tested?

  • npm run check — typecheck + lint pass.
  • npm test — 274/274 tests pass (6 new vitest cases for this scenario, covering no-client, compliant-on-dated, compliant-on-draft, $schema/$defs strip, SEP-2106 strip on draft, and a round-trip-fidelity sanity check).
  • End-to-end against the bundled reference client (CONTRIBUTING.md "Running your scenario" form):
$ npm run build
$ node dist/index.js client \
  --command "npx tsx examples/clients/typescript/everything-client.ts" \
  --scenario json-schema-2020-12-preservation
Starting scenario: json-schema-2020-12-preservation
Checks:
2026-06-05T03:25:14.074Z [json-schema-2020-12-client-tool-found                    ] SUCCESS Client called tools/list and
the mock server advertised 'json_schema_2020_12_tool'
2026-06-05T03:25:14.074Z [json-schema-2020-12-client-echo-completed                ] SUCCESS Client called tools/call
'json_schema_echo' with a 'schema' object argument
2026-06-05T03:25:14.074Z [json-schema-2020-12-client-$schema-preserved             ] SUCCESS Client preserved $schema =
'https://json-schema.org/draft/2020-12/schema'
2026-06-05T03:25:14.074Z [json-schema-2020-12-client-$defs-preserved               ] SUCCESS Client preserved $defs with the
expected nested definitions
2026-06-05T03:25:14.074Z [json-schema-2020-12-client-additionalProperties-preserved] SUCCESS Client preserved
additionalProperties = false on the focal inputSchema
2026-06-05T03:25:14.074Z [sep-2106-client-composition-keywords-preserved           ] SUCCESS Client preserved composition
keywords (allOf/anyOf) on the focal inputSchema
2026-06-05T03:25:14.074Z [sep-2106-client-conditional-keywords-preserved           ] SUCCESS Client preserved conditional
keywords (if/then/else) on the focal inputSchema
2026-06-05T03:25:14.074Z [sep-2106-client-anchor-keyword-preserved                 ] SUCCESS Client preserved $anchor inside
$defs.address

Test Results:
Passed: 8/8, 0 failed, 0 warnings

The negative case (a client that strips $schema/$defs before echoing) is covered by src/scenarios/client/json-schema-2020-12-preservation.test.ts rather than by a separate broken example client, matching the pattern used by json-schema-ref-deref.test.ts.

src/seps/traceability.json will drift after this change picks up the six new requirement rows in sep-2106.yaml; per AGENTS.md the traceability workflow will refresh it via a follow-up PR rather than as part of this one.

Breaking Changes

None.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

The mock server, using the version-aware `MockServer` abstraction from modelcontextprotocol#321,
first advertises a focal tool with a detailed inputSchema and also offers a
permissive `json_schema_echo` tool. The client under test retrieves the
inputSchema via `tools/list`, then sends the same inputSchema back to the server
using the echo tool, completing the round-trip.

The scenario diffs from the echoed schema against the canonical fixture. It
emits eight per-keyword conformance checks. SEP-2106 vocabulary checks are
soft-gated using the existing sep2106KeywordCheckStatus helper. They only flip
to FAILURE during DRAFT-2026-v1 runs.

Signed-off-by: Satoshi Ito <satoshi.ito.tf@hitachi.com>
@pkg-pr-new

pkg-pr-new Bot commented Jun 9, 2026

Copy link
Copy Markdown

Open in StackBlitz

npx https://pkg.pr.new/@modelcontextprotocol/conformance@335

commit: 1a82ca9

The stateful test helper passed createServerStateful bare, which no longer
compiles now that it takes a second specVersion argument, and forcing the
stateful mock at the draft version trips wire-schema validation (draft
results require the SEP-2549 resultType/ttlMs/cacheScope members). Use
testScenarioContext(DRAFT_PROTOCOL_VERSION), which selects the stateless
mock server, and drive it with sendStatelessRequest, matching the pattern
in json-schema-ref-deref.test.ts.
SEP-2106 §1 contains no normative preservation sentence, so the six new
sep-2106.yaml rows should not attribute a spec MUST to it. Reword the
remaining rows as expectations derived from issue modelcontextprotocol#101. Also drop the
three rows whose check IDs are not sep-prefixed: the traceability
manifest only collects IDs matching /^sep-\d+-/, so those rows could
never be marked tested.
The draft protocol identifier was renamed in modelcontextprotocol#331. Interpolate
DRAFT_PROTOCOL_VERSION into the SKIPPED errorMessage suffix (matching
the server-side scenario) and update the header comment.
@pcarleton

Copy link
Copy Markdown
Member

Thanks @pugsatoshi — the scenario is solid and runs green end-to-end at both 2025-11-25 and the draft spec version, including the wire-schema validation from #399. Main moved under the PR, so I pushed the fixes onto your branch rather than a review round trip:

  1. Merge with current main; the draft-target unit tests now drive the stateless lifecycle (post-Validate all wire messages against the per-version spec JSON schema #399 the stateful mock requires the SEP-2549 result fields) and the test helper matches createServerStateful's new signature.
  2. sep-2106.yaml: reworded the preservation rows as derived-from-Consider adding client test for JSON Schema 2020-12 preservation (SEP-1613) #101 expectations (SEP-2106 has no normative preservation MUST), and dropped three rows whose check IDs the traceability collector can't pick up.
  3. Replaced stale DRAFT-2026-v1 references with 2026-07-28.

Verified: 489/489 unit tests, scenario 9/9 end-to-end at both spec versions. We also prototyped the missing ~10-35 line fixture handler in all five Tier 1 SDK conformance clients — every SDK passes 9/9 (no SDK strips any keyword), so SDK repos will get those handler patches as follow-up PRs; until then a new-scenario failure can be baselined via expected-failures.

@pcarleton pcarleton left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed, fixes applied on-branch, all five Tier 1 SDKs verified 9/9 with the fixture handler prototypes.

@pcarleton
pcarleton enabled auto-merge (squash) July 31, 2026 17:23
@pcarleton
pcarleton merged commit 81eb1c3 into modelcontextprotocol:main Jul 31, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Consider adding client test for JSON Schema 2020-12 preservation (SEP-1613)

2 participants