Skip to content

[Product Gap] Bound HTTP/1.1 semantics over the authenticated TLS stream #9

Open
Open
[Product Gap] Bound HTTP/1.1 semantics over the authenticated TLS stream#9
Labels
bugSomething isn't workingpriority: hightype: bug

Description

@seonghobae
seonghobae
opened on Aug 7, 2026

Buyer-visible problem

OriginWeave can now authorize a canonical origin, bind it to an approved resolved destination, prove the exact operating-system TCP peer, and authenticate the requested HTTPS service on that same stream. It still cannot safely perform an HTTP exchange. A buyer-visible navigation path remains vulnerable to ambiguous message framing, unbounded headers or content, compression bombs, method/status mistakes, incomplete responses, integrity mismatches, MIME confusion, unsafe filenames, and redirects that bypass the existing origin/destination/transport/TLS authorities.

Required bounded vertical slice

Create an independently reusable Rust originweave-http crate that performs one bounded HTTP/1.1 request/response exchange over an existing AuthenticatedTlsConnection. It must never resolve a hostname, open or reconnect a socket, select a proxy, parse PAC, weaken TLS, control Chromium, or persist a download directly.

HTTP/2 and HTTP/3 remain separate later adapters. The first slice supports only a negotiated http/1.1 ALPN value or an explicitly documented direct-test policy where ALPN absence is permitted; it must reject h2, h3, and any other application protocol.

Request authority

  • Accept the authenticated TLS stream and immutable TLS evidence, the exact canonical request target, one bounded HTTP policy, and an explicit method.
  • Require the request origin to equal the authenticated TLS origin.
  • Permit only an explicit bounded method allow-list. The first slice should support GET and HEAD; state-changing methods remain a later action-policy integration.
  • Generate the request target and Host field from the canonical origin and target contract; do not accept caller-supplied framing, connection, proxy, authorization, cookie, or hop-by-hop fields in the general header collection.
  • Reject control characters, obsolete line folding, whitespace before field names, invalid field names, invalid field values, duplicate singleton fields, excessive field count, excessive field-name/value bytes, and total header-section overflow before emitting bytes.
  • Never place credential values in evidence, errors, logs, or model-visible output.

HTTP/1.1 framing

Implement RFC 9112 message parsing and framing fail closed:

  • strict CRLF start-line and field-line handling;
  • bounded status-line and header-section parsing;
  • exact response body-length determination from request method, response status, Transfer-Encoding, Content-Length, and connection close;
  • reject a message containing both Transfer-Encoding and Content-Length;
  • reject unsupported transfer codings, invalid chunk sizes, chunk extensions outside a small explicit policy, malformed trailers, conflicting or invalid Content-Length, incomplete bodies, premature EOF, and bytes that would make the next-message boundary ambiguous;
  • treat HEAD, 1xx, 204, 304, successful CONNECT, and other no-content semantics according to RFC 9110/9112 rather than merely following header presence;
  • disable connection reuse in the first slice so parser state cannot leak across requests.

Resource budgets

Validate every limit before use and enforce a single monotonic exchange deadline:

  • request bytes;
  • status-line bytes;
  • header-field count;
  • one field-name and field-value size;
  • total header-section bytes;
  • chunk count;
  • encoded content bytes;
  • decoded content bytes;
  • content-coding expansion ratio;
  • trailer count and bytes;
  • interim response count;
  • total elapsed time and per-read/per-write blocking;
  • redirect metadata retained for the caller.

Content codings must stream through bounded decoders. No implementation may allocate from untrusted declared lengths, buffer an entire unbounded response, or retry beyond the total deadline. When a limit is crossed, stop reading, close the owned stream, and return a typed failure without exposing a partial authenticated response as complete.

Semantics and redirects

  • Retain the exact status code and reason-phrase bytes separately; do not derive behavior from the reason phrase.
  • Return redirect evidence, not an automatically followed redirect.
  • A caller must submit every redirect target through canonical origin parsing, destination approval, direct TCP binding, TLS authentication, and a new HTTP exchange plan.
  • Preserve secure-scheme downgrade, redirect-cycle, target digest, hop-limit, capability, and action-risk checks as separate existing authority boundaries.
  • Make response completeness explicit. A partial or truncated response cannot be represented as a successful complete response.

Integrity fields

Support optional RFC 9530 validation after exact message framing:

  • parse Content-Digest and Repr-Digest as HTTP Structured Fields;
  • support an explicit algorithm allow-list, initially sha-256 and sha-512 only;
  • distinguish absent, verified, unsupported, malformed, and mismatched integrity evidence;
  • never claim malicious-tamper protection from an unsigned digest alone;
  • record that RFC 9530 obsoletes RFC 3230 and the legacy Digest/Want-Digest fields.

MIME and download handoff

  • Parse the supplied Content-Type with a deterministic MIME record rather than string-prefix heuristics.
  • Compute an observed MIME type from a bounded resource header using the current WHATWG MIME Sniffing contract or a separately versioned compatible adapter.
  • Preserve supplied MIME, computed MIME, X-Content-Type-Options no-sniff state, and mismatch classification as evidence.
  • Treat HTML, XML, JavaScript, SVG, and PDF as scriptable/high-risk content classes for downstream policy.
  • Do not infer MIME from a URL extension.
  • Parse Content-Disposition into a safe metadata record, but do not create files. Normalize or reject control characters, path separators, absolute paths, device names, ambiguous Unicode, excessive names, and extension/type mismatch before any download adapter can persist bytes.
  • Pass bytes and evidence to a later download/MIME authority; this crate must not execute, render, extract, or save content.

Evidence and errors

Emit immutable, credential-free HttpExchangeEvidence containing at minimum:

  • authenticated canonical origin and exact TCP peer identifiers inherited from TLS evidence;
  • request method and canonical target digest without query credential retention;
  • HTTP version and status code;
  • bounded response-header names plus allow-listed non-sensitive values only;
  • encoded and decoded byte counts;
  • transfer/content coding decisions;
  • response completeness;
  • supplied and computed MIME types, no-sniff state, and mismatch classification;
  • integrity-field status and algorithm identifiers;
  • redirect target digest and canonical origin when present, with sensitive query values excluded;
  • exchange duration and configured budgets;
  • source hashes sufficient for replay and audit without retaining secrets.

Public errors must implement deterministic Display and std::error::Error, preserve underlying I/O/decode failures as sources, and distinguish invalid policy, request authority mismatch, invalid request fields, malformed status/headers, framing ambiguity, unsupported transfer/content coding, incomplete response, timeout, each resource-budget violation, integrity mismatch, invalid MIME, unsafe disposition metadata, unexpected ALPN, and peer/TLS evidence mismatch.

Realistic tests

Use real loopback HTTP/1.1 servers over the authenticated TLS test authority. Cover at least:

  • bounded GET with Content-Length success;
  • bounded HEAD and no-content status semantics;
  • valid chunked content and trailers;
  • conflicting Content-Length values;
  • Transfer-Encoding plus Content-Length rejection;
  • whitespace-before-field-name and obsolete folding rejection;
  • malformed status lines, invalid chunk syntax, premature EOF, and incomplete TLS close;
  • oversized line, field count, header section, body, decoded body, trailer, chunk count, and interim response count;
  • gzip/deflate expansion-ratio enforcement with streamed decoding;
  • hard total deadline before and after each blocking operation;
  • Content-Digest/Repr-Digest valid, absent, unsupported, malformed, and mismatch cases;
  • supplied/computed MIME agreement, mislabeled scriptable content, no-sniff behavior, ZIP/PDF/HTML signatures, and extension spoofing;
  • safe and hostile Content-Disposition names across Unix and Windows path conventions;
  • redirect returned as evidence without follow-up network I/O;
  • exact proof that no reconnect, DNS, proxy, PAC, Chromium, unrestricted logging, credential retention, or file persistence path exists;
  • exact 100% production function, line, region, and branch coverage and complete public rustdoc.

Architecture and documentation

  • Add an ADR separating HTTP semantics from origin parsing, destination approval, TCP peer proof, TLS identity, proxy/PAC routing, redirect authorization, MIME/download persistence, Chromium integration, and content execution.
  • Update AGENTS.md, CLAUDE.md, ARCHITECTURE.md, README.md, CHANGELOG.md, docs/product-roadmap.md, docs/quality-gates.md, and docs/doctoring.md.
  • Record all adopted limits as product safety budgets, not protocol-validity claims.
  • Add a Mermaid sequence diagram from authenticated TLS stream through bounded exchange evidence and redirect reauthorization.
  • Preserve independent crate usability for OriginWeave, naruon, or another CWL service.

Commercial proof

A deterministic loopback suite demonstrates that the same authenticated TLS stream yields a complete HTTP response only when message framing is unambiguous, all resource budgets hold, optional integrity evidence verifies, and content metadata is safely classified. Redirects are surfaced but never followed without repeating the full authority chain.

Standards references — APA 7th

Fielding, R., Nottingham, M., & Reschke, J. (2022). HTTP semantics (RFC 9110). Internet Engineering Task Force. https://doi.org/10.17487/RFC9110

Fielding, R., Nottingham, M., & Reschke, J. (2022). HTTP/1.1 (RFC 9112; STD 99). Internet Engineering Task Force. https://doi.org/10.17487/RFC9112

Polli, R., & Pardue, L. (2024). Digest fields (RFC 9530). Internet Engineering Task Force. https://doi.org/10.17487/RFC9530

WHATWG. (2026, July 17). MIME sniffing: Living Standard. https://mimesniff.spec.whatwg.org/

Out of scope

HTTP/2, HTTP/3, proxy/PAC, connection pooling, caching, cookies, authentication, client certificates, WebSocket, WebTransport, range assembly, multipart form upload, automatic redirect following, file persistence, archive extraction, active-content rendering, Chromium Network Service integration, and browser UI remain later independently reviewable slices.

Activity

  1. seonghobae commented on Aug 7, 2026

    @seonghobae
    seonghobae
    on Aug 7, 2026 – with ChatGPT Codex Connector
    ContributorAuthor

    Standards addendum for the implementation spec

    Current primary-source review adds two normative parsing contracts that should be explicit before implementation:

    1. Parse Content-Digest and Repr-Digest with an RFC 9651 Structured Fields implementation, while retaining the field-specific dictionary and byte-sequence constraints defined by RFC 9530. RFC 9651 obsoletes RFC 8941 and is backward-compatible with fields defined against the earlier revision; malformed Structured Fields must be rejected or ignored only according to the strict RFC 9651 and field-specific policy, never partially recovered.
    2. Parse HTTP Content-Disposition according to RFC 6266, and decode filename* extended parameters according to RFC 8187, which obsoletes RFC 5987. When both filename and filename* are present, prefer the valid filename* value, but continue to treat every server-provided filename as advisory and apply the existing fail-closed path, device-name, control-character, Unicode, and extension/MIME safety policy before any persistence handoff.

    RFC 9530 currently has verified editorial errata; implementation tests should use the normative RFC text and maintain an errata review record without treating editorial corrections as protocol changes.

    Additional APA 7th references

    Nottingham, M., & Kamp, P.-H. (2024). Structured field values for HTTP (RFC 9651). Internet Engineering Task Force. https://doi.org/10.17487/RFC9651

    Reschke, J. F. (2011). Use of the Content-Disposition header field in the Hypertext Transfer Protocol (HTTP) (RFC 6266). Internet Engineering Task Force. https://doi.org/10.17487/RFC6266

    Reschke, J. F. (2017). Indicating character encoding and language for HTTP header field parameters (RFC 8187). Internet Engineering Task Force. https://doi.org/10.17487/RFC8187

  2. seonghobae commented on Aug 7, 2026

    @seonghobae
    seonghobae
    on Aug 7, 2026 – with ChatGPT Codex Connector
    ContributorAuthor

    Reviewable implementation sequence

    To avoid another oversized authority-boundary PR while still producing usable, fail-closed increments, implement the issue as four ordered changes on one independently reusable originweave-http crate. Each change must keep the workspace green, preserve the authenticated TLS stream as the only transport authority, and end with exact 100% production function/line/region/branch coverage.

    1. Request authority and fixed-length response kernel

    • Introduce HttpMethod, canonical request-target/Host derivation, bounded policy types, forbidden hop-by-hop/credential field validation, deterministic request serialization, strict status/header parsing, GET/HEAD, no-content status semantics, exact Content-Length, and explicit Connection: close ownership.
    • Permit only authenticated http/1.1 or an explicit ALPN-absence direct-test policy; reject every other negotiated protocol.
    • Support one complete fixed-length or semantically bodyless exchange over the supplied AuthenticatedTlsConnection; no DNS, reconnect, proxy/PAC, redirects, persistence, content coding, or MIME inference.
    • Real TLS loopback tests: bounded GET, HEAD, 1xx/204/304 handling, origin mismatch, invalid request fields, whitespace-before-field-name, obs-fold, conflicting Content-Length, premature EOF, status/header/body budgets, total deadline, and exact proof of no second socket.

    2. Complete RFC 9112 framing

    • Add interim-response bounds, chunked coding, bounded chunk extensions, trailers, close-delimited bodies, Transfer-Encoding/Content-Length ambiguity rejection, unsupported transfer-coding rejection, and exact next-message-boundary checks.
    • Keep connection reuse disabled; every successful or failed exchange consumes and closes the stream.
    • Real TLS loopback tests cover valid chunks/trailers, invalid chunk syntax, TE+CL, malformed trailers, too many chunks/interims, incomplete close, trailing ambiguous bytes, and deadline enforcement before and after every blocking operation.

    3. Bounded content codings and digest evidence

    • Add streaming gzip/deflate decoding with encoded/decoded/ratio budgets and no allocation from declared untrusted lengths.
    • Add RFC 9530 Content-Digest/Repr-Digest verification using RFC 9651 Structured Fields, with explicit sha-256/sha-512 allow-lists and absent/verified/unsupported/malformed/mismatch evidence.
    • Tests cover compressed expansion bombs, truncation, unsupported coding, chained coding policy, valid and mismatched digests, malformed dictionaries/byte sequences, duplicate algorithms, and the distinction between content and representation digest domains.

    4. MIME, disposition, redirect, and audit handoff

    • Add deterministic supplied MIME parsing, a versioned bounded WHATWG MIME-sniff adapter, no-sniff and scriptable-content classification, RFC 6266/RFC 8187 disposition parsing and filename safety, and redirect evidence without network follow-up.
    • Add immutable credential-free HttpExchangeEvidence, deterministic errors with preserved sources, architecture/ADR/doctoring/roadmap/quality-gate updates, and the Mermaid authority-chain sequence.
    • Tests cover supplied/observed MIME agreement and mismatch, no-sniff, HTML/XML/JS/SVG/PDF/ZIP signatures, extension spoofing, Unix/Windows hostile names, filename* precedence, redirect evidence, query redaction, and proof that no file write/render/extract/execute path exists.

    The first implementation PR should be stacked only after the TLS PR’s exact head is stable and should target the TLS branch until that prerequisite merges; reviews, approvals, and checks must not transfer when it is retargeted to main.

  3. seonghobae
    added
    priority: high
    bugSomething isn't working
    type: bug
    on Sep 7, 2026

  4. seonghobae commented on Sep 22, 2026

    @seonghobae
    seonghobae
    on Sep 22, 2026 – with ChatGPT Codex Connector
    ContributorAuthor

    2026-09-23 current repair evidence for canonical HTTP authority: PR #37 is now at exact 15e7c97dc1102934fa1e3ca8517b86938a808565, ordinary-forward on protected main@87c4daa1830bac5a5228b6036752ad5633232085.

    Predecessor 367e9f62714215b032ae10df6fea730b7e254a56 finally cleared hosted-runner admission and produced a real repository-contract RED. CI 35778361555 admitted both jobs: Production coverage 106917229967 completed SUCCESS, while Rust contracts 106917229535 failed at Check Python repository contracts before formatting/tests/Clippy/rustdoc. SAST 35778361885 independently completed SUCCESS. RCA found no HTTP/runtime defect: tests/test_http_content_coding_documentation_contract.py required lowercase literal stacked content codings, while ADR 0011 correctly begins the sentence Stacked content codings .... The contract was coupled to prose capitalization rather than the semantic interoperability restriction.

    Minimal ordinary-forward repair 15e7c97d... changes only that assertion to search adr.lower(). Production Rust, decoder semantics, design/ADR wording, dependencies, workflows, rulesets, and coverage denominators are unchanged. Fresh exact-head runs are live and nonterminal: CI 35789840659, SAST 35789840604, Security 35789840636, CodeQL 35789840612. The earlier central queue report was corrected at .github#712 comment 5784887730; no blind rerun or source-neutral wake was used.

    Issue #9 remains open until one unchanged exact head receives native repository/security evidence, eligible independent approval, all required central verdicts, and normal protected-main integration. Predecessor workflow/review results do not transfer.

  5. seonghobae commented on Sep 22, 2026

    @seonghobae
    seonghobae
    on Sep 22, 2026 – with ChatGPT Codex Connector
    ContributorAuthor

    PR #37 HTTP/1.1 reconstruction has a new standards-conformance repair on the ordinary-forward branch.

    Fresh review found that the Content-Encoding: gzip path treated a second valid gzip member as trailing garbage. That is stricter than the coding it advertises: RFC 9110 defines HTTP gzip by reference to RFC 1952, and RFC 1952 defines a gzip file as a series of concatenated members.

    Repair lineage:

    • test-first da736d43153bef1d3488aa8df389a4200a0ca5b8: adds a regression requiring two independently encoded RFC 1952 members to decode as one representation; hosted workflows for this intermediate head were superseded/cancelled before an executable RED completed, so no hosted RED is claimed.
    • causal fix 5994acab7e58a76edd9cc80800c7dfc8acca0fd6: replaces the single-member bufread::GzDecoder path with bufread::MultiGzDecoder, preserving the existing aggregate decoded-size and expansion-ratio limits. Concatenated members are accepted; malformed non-gzip trailing bytes still fail as ContentDecodingFailed.
    • doctoring 28321dec3a812ff63886de8d33fb102d4894a3fe: records RFC 1952/RFC 9110 authority and also removes a stale Content-Length sentinel/EOF statement that contradicted the already-landed self-delimiting Content-Length repair.

    Issue #9 remains open. No protected-main shipment is claimed until the exact current head receives its required CI/security/CodeQL/review evidence and normal integration.

  6. seonghobae commented on Sep 22, 2026

    @seonghobae
    seonghobae
    on Sep 22, 2026 – with ChatGPT Codex Connector
    ContributorAuthor

    PR #37 ordinary-forward currentness update: exact head is now 2f9ab91e28b0570e69fbb00ba011baa3d8b02e66 on unchanged protected main@87c4daa1830bac5a5228b6036752ad5633232085.

    The remaining documentation-to-code gap from the RFC 1952 repair is now closed: 2f9ab91e... adds an Unreleased/Changed CHANGELOG entry stating that gzip accepts concatenated RFC 1952 members through the bounded multi-member decoder while retaining aggregate decoded-size/expansion-ratio limits and typed failure for malformed trailing data. Runtime behavior is unchanged from 5994acab...; this is documentation currentness only.

    The immediately superseded 28321dec... exact-head workflows never reached executable steps: CI Rust/coverage, Semgrep, Security scope detection, and CodeQL language detection were all runner_id=0, steps=[] queue-admission waits. That exact specimen is recorded at canonical queue owner .github#712 comment 5782294990; predecessor queue evidence does not transfer to 2f9ab91e.... Fresh exact-head workflows/review remain mandatory before normal integration.

  7. seonghobae commented on Sep 22, 2026

    @seonghobae
    seonghobae
    on Sep 22, 2026 – with ChatGPT Codex Connector
    ContributorAuthor

    PR #37 current authority advanced ordinary-forward to 4869038c23f4fecb88a8b5d36b980f6a800fccaa. Fresh compare from doctoring head 28321dec... is 2 commits ahead / 0 behind with an aggregate delta of exactly one added CHANGELOG.md line: the RFC 1952 multi-member gzip release note. 4869038c... only restores the terminal newline that the preceding full-file Contents API write had unintentionally removed.

    Fresh exact-head CI/SAST/Security/CodeQL generations all materialized but remain blocked before executable steps on runner admission: CI 35772008929, SAST 35772008869, Security 35772008659, CodeQL 35772008774. The exact specimen is recorded at canonical queue owner .github#712 comment 5782414546. No current-head GREEN, source failure, independent approval, or merge acceptance is claimed.

  8. seonghobae commented on Sep 22, 2026

    @seonghobae
    seonghobae
    on Sep 22, 2026 – with ChatGPT Codex Connector
    ContributorAuthor

    Current #37 HTTP authority update on exact 367e9f62714215b032ae10df6fea730b7e254a56 (base main@87c4daa1830bac5a5228b6036752ad5633232085). Fresh code/docs review found the approved design and Proposed ADR lagging the already-repaired decoder: production accepts concatenated RFC 1952 gzip members via MultiGzDecoder and records bounded raw RFC 1951 DEFLATE fallback as DeflateRawCompatibility, but the design still said raw deflate was rejected and the ADR still described zlib-only deflate. Test-first documentation contracts were added (1b3d569b..., then ADR coverage 6d6e8c09...); ordinary-forward repairs fdbdad4b... and 367e9f62... make design + ADR match executable behavior and preserve RFC 9110/RFC 1951/RFC 1952 boundaries. No HTTP production code changed in this repair.

    The same standards review makes the next buyer-visible interoperability gap explicit rather than hiding it: RFC 9110 permits an ordered list of content codings, while the bounded first slice intentionally rejects stacked codings. Safe expansion needs a reviewed maximum coding depth, reverse-order decoding, cumulative decoded-byte/expansion accounting across the whole chain, evidence-schema support, and realistic TLS loopback fixtures including malformed inner layers and expansion attacks. This remains follow-up work; it is not justification to widen #37 while exact-head gates are unresolved.

    Current exact-head CI 35778361555, SAST 35778361885, Security 35778361631, and CodeQL 35778361641 are all still pre-step queued. The actionable jobs have no runner and steps=[]; Security gitleaks is conditionally skipped while scope detection is unresolved, so no hosted GREEN is claimed. Exact specimen is recorded at canonical runner-admission owner .github#712 comment 5783349121. Keep #9 open through normal protected integration; no predecessor check/review transfers to this head.

  9. seonghobae commented on Sep 23, 2026

    @seonghobae
    seonghobae
    on Sep 23, 2026 – with ChatGPT Codex Connector
    ContributorAuthor

    Canonical HTTP authority #37 is now exact 7007140e95f94d1601259384bf646b8bde6a240c. Predecessor 15e7c97... finally received a hosted runner: exact production coverage is GREEN, while Rust contracts failed specifically at rustfmt after Python repository contracts passed. The uploaded canonical rustfmt artifact identified only three formatting deltas (two missing terminal newlines and one import layout); commit 7007140e... applies that artifact verbatim with no runtime semantic change. Fresh CI/SAST/Security/CodeQL generations exist and predecessor evidence does not transfer. Stacked-coding successor #327 has non-force adopted the repaired parent at b7e056a781297d570c908710737028ecd8ed5abd and remains test-only pending executable RED.

  10. seonghobae commented on Sep 23, 2026

    @seonghobae
    seonghobae
    on Sep 23, 2026 – with ChatGPT Codex Connector
    ContributorAuthor

    HTTP authority currentization: canonical parent #37 is now exact 219b43bfa87ab4fd90a77bb962df2797925bf661. Predecessor 7007140e... did run: Production coverage was SUCCESS, while Rust contracts reached strict Clippy after repository contracts/format/tests passed and failed only on four expect_used plus one panic in the chunked committed-prefix regression. Ordinary-forward test-only repairs da7ffb29189b089d536efd925df4de2e36d3b402 and 219b43bfa87ab4fd90a77bb962df2797925bf661 remove those violations without suppressions or gate changes. Fresh exact CI 35816715517 is non-terminal, so #9 remains open and no predecessor GREEN is promoted.

    The stacked-coding successor #327 has been non-force restacked onto this exact parent at child beabc7d3e27c5c407d2847ff179eb16682ad86cd; compare is 5 ahead / 0 behind and still only one +235-line integration-test delta. Production stacked decoding is still withheld until that exact child produces its intended executable RED.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority: hightype: bug

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions