Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
145 commits
Select commit Hold shift + click to select a range
d177545
build(chtypes): build, package and release against the chtypes SDK
EricAndrechek Oct 1, 2026
e58d17f
docs: describe the chtypes type layer on the multi-tenant text
EricAndrechek Oct 1, 2026
b4779b2
Merge docs for the chtypes migration into chtypes-v2
EricAndrechek Oct 1, 2026
e071158
feat(policy): resolved row-filter predicates and a typed WHERE renderer
EricAndrechek Oct 1, 2026
6e1fce2
feat(discovery): refresh hooks before loaded, and the server zone
EricAndrechek Oct 1, 2026
3f27439
feat(typelayer): tenant-keyed chtypes engine with lazy handle pools
EricAndrechek Oct 1, 2026
443b19a
Merge the Go foundation for the chtypes migration into chtypes-v2
EricAndrechek Oct 1, 2026
4995331
feat(api)!: serve /v1/query and pipes through ClickHouse's HTTP inter…
EricAndrechek Oct 1, 2026
e000751
Merge branch 'chtypes-v2' into v2-w3
EricAndrechek Oct 1, 2026
6dd65fb
feat(typelayer): parse a row under the INSERT column list it was publ…
EricAndrechek Oct 1, 2026
35a008e
feat(query): bind integer policy claims through the strict cast
EricAndrechek Oct 1, 2026
2aa6f7d
feat(config,ingest): chtypes registry key and typelayer insert settings
EricAndrechek Oct 1, 2026
e87c9ce
Merge the HTTP read path for the chtypes migration into chtypes-v2
EricAndrechek Oct 1, 2026
b02bdd2
Merge branch 'chtypes-v2' into v2-w5
EricAndrechek Oct 1, 2026
852d5f8
feat(stream): judge row-level security with the type layer, per tenant
EricAndrechek Oct 1, 2026
91e86dc
Merge the chtypes-backed stream filter into chtypes-v2
EricAndrechek Oct 1, 2026
0537e89
feat(ingest)!: judge every body with ClickHouse's own parser
EricAndrechek Oct 1, 2026
3ba7c6e
Merge branch 'chtypes-v2' into v2-w5
EricAndrechek Oct 1, 2026
6fec76e
Merge branch 'chtypes-v2' into v2-w2
EricAndrechek Oct 1, 2026
1ceb21d
test(e2e)!: pin ClickHouse's own ingest verdicts and exception_code
EricAndrechek Oct 1, 2026
0499111
Merge the chtypes-judged ingest path into chtypes-v2
EricAndrechek Oct 1, 2026
8dcb496
Merge branch 'chtypes-v2' into v2-w5
EricAndrechek Oct 1, 2026
a776b2c
feat(app): open the type layer in API processes and bind it per tenant
EricAndrechek Oct 1, 2026
df6f93d
build(chtypes): move to SDK go/v0.5.1 and the ClickHouse 26.8 line
EricAndrechek Oct 1, 2026
3694f5e
fix(typelayer): set each library's zone through the SDK default
EricAndrechek Oct 1, 2026
cd1ddd7
fix(typelayer): compile with ClickHouse's type gates
EricAndrechek Oct 1, 2026
e1e3404
Merge branch 'chtypes-v2' into v2-w1c
EricAndrechek Oct 1, 2026
92c6c6e
test(integration): chtypes on the app harness, with policies and tokens
EricAndrechek Oct 1, 2026
9f78c0b
Merge SDK go/v0.5.1 and the ClickHouse 26.8 line into chtypes-v2
EricAndrechek Oct 1, 2026
2f230f8
Merge branch 'chtypes-v2' into v2-w5
EricAndrechek Oct 1, 2026
eb5ba5b
test(ingest): feed DateTime64 the way ClickHouse 26.8 reads a number
EricAndrechek Oct 1, 2026
be20317
Merge app wiring, config and the integration suite into chtypes-v2
EricAndrechek Oct 1, 2026
99a4529
refactor: delete the Go-side canonicalizer, row evaluator and compact…
EricAndrechek Oct 1, 2026
eb36644
Merge the old-API sweep into chtypes-v2
EricAndrechek Oct 1, 2026
6a72536
docs: reconcile docs with the chtypes type layer as landed
EricAndrechek Oct 1, 2026
4c9786d
Merge the docs reconcile pass into chtypes-v2
EricAndrechek Oct 1, 2026
c6b0128
fix(query): parse timestamp filters in ClickHouse, send in lists as t…
EricAndrechek Oct 1, 2026
3858fb7
docs(query): timestamp filters parse in ClickHouse; in lists are unca…
EricAndrechek Oct 1, 2026
9a044a0
Merge branch 'chtypes-v2' into v2-w3b
EricAndrechek Oct 1, 2026
39d3528
docs(api): give the timestamp-filter rule its own paragraph
EricAndrechek Oct 1, 2026
f32dd7a
Merge ClickHouse-parsed timestamp filters and external-data in-lists …
EricAndrechek Oct 1, 2026
943a828
docs(access-control): timestamp claims take the zone-less form
EricAndrechek Oct 1, 2026
9513fdb
test(integration): fit the suite back under its timeout
EricAndrechek Oct 1, 2026
e2e057e
Merge branch 'chtypes-v2' into v2-integ
EricAndrechek Oct 1, 2026
6915489
test(integration): run internal/api's live filter test in the target
EricAndrechek Oct 1, 2026
67e2b5d
build(chtypes): move to SDK go/v0.5.2
EricAndrechek Oct 1, 2026
d0ba5de
fix(discovery): publish refreshes in the order they started
EricAndrechek Oct 1, 2026
2dd4de7
test(integration): clear gosec's G703 on the settings cleanup
EricAndrechek Oct 1, 2026
d02cb78
docs: fix review-round findings across the chtypes docs
EricAndrechek Oct 1, 2026
5aa1b81
test(typelayer): move the test engine helpers into typelayertest
EricAndrechek Oct 1, 2026
db21526
fix(api): cap query-path connections per pool, not per server
EricAndrechek Oct 1, 2026
3c6d674
perf(typelayer): grow a role shape's handle pool under contention
EricAndrechek Oct 1, 2026
60862a8
chore(api): note that the 64 MiB response cap covers every query path
EricAndrechek Oct 1, 2026
8464208
fix(api): render DateTime as RFC 3339 UTC on every read and stream su…
EricAndrechek Oct 1, 2026
a888661
fix(sdk): compare timestamps as instants in stream filters and liveQuery
EricAndrechek Oct 1, 2026
3662865
Merge per-tenant read cap and RFC 3339 UTC rendering
EricAndrechek Oct 1, 2026
b708c1f
Merge round-1 docs review fixes
EricAndrechek Oct 1, 2026
4aa01e3
fix(ingest): role shapes keep denied columns, stamp _eq columns, take…
EricAndrechek Oct 1, 2026
b6035d3
fix(ingest): refuse content after a JSON array body
EricAndrechek Oct 1, 2026
3c8542a
test(integration): stored rows for denied, stamped and ephemeral columns
EricAndrechek Oct 1, 2026
c88603a
Merge round-1 code review fixes (insert column policy, EPHEMERAL, rol…
EricAndrechek Oct 1, 2026
330fd09
test(typelayer): use the in-package engine helper
EricAndrechek Oct 1, 2026
3674b79
docs: match the role-shape, EPHEMERAL, timestamp and read-cap fixes
EricAndrechek Oct 1, 2026
9dde7ed
docs: fix round-2 review findings (RFC 3339 examples, wire columns, r…
EricAndrechek Oct 1, 2026
6221019
docs: tighten EPHEMERAL, timestamp rendering and check-literal claims
EricAndrechek Oct 1, 2026
dc9ee8f
chore: label and coverage-exclude the typelayer package
EricAndrechek Oct 1, 2026
eab6e3d
chore: correct comments the type-layer migration left stale
EricAndrechek Oct 1, 2026
9da80ec
docs: fix round-3 findings (Nullable nulls, dedupe example, stale mec…
EricAndrechek Oct 1, 2026
0343354
fix(ingest): blank layout newlines after a JSON array's closing bracket
EricAndrechek Oct 1, 2026
56ba58f
chore: correct comments contradicted by measurement on 26.8
EricAndrechek Oct 1, 2026
4d35acf
docs: correct ingest codes, timestamp claims and stale mechanisms
EricAndrechek Oct 1, 2026
4950a59
ci: the fetch script takes no --frozen flag; it always fetches frozen
EricAndrechek Oct 1, 2026
289d55a
docs: make dev OTel endpoint, operator-key scope, ClickHouse statuses
EricAndrechek Oct 1, 2026
e7215e8
perf(typelayer): compile role shapes outside the role cache lock
EricAndrechek Oct 1, 2026
f367511
perf(typelayer): give every handle the whole filter cache budget
EricAndrechek Oct 1, 2026
6cb72bd
chore: reflow comments left over-long by the last round
EricAndrechek Oct 1, 2026
e8ad99a
docs: name roweval.go in the stream package and who sets the e2e OTLP…
EricAndrechek Oct 1, 2026
f09d96f
docs: show ClickHouse's escaped slashes in raw stream and pipe output
EricAndrechek Oct 1, 2026
417d7b0
fix(api): leave / unescaped wherever ClickHouse renders JSON
EricAndrechek Oct 1, 2026
e219066
test(stream): pin that an unreadable row is declined, not an error
EricAndrechek Oct 1, 2026
7ad1be8
docs: who labels each withheld row, and what the type layer holds in …
EricAndrechek Oct 1, 2026
871c86c
docs: the envelope re-encodes the writer's row without changing any v…
EricAndrechek Oct 1, 2026
241f869
docs: finish the exact-bytes correction; say what the error reason means
EricAndrechek Oct 1, 2026
da0c0aa
docs: the last two copies of the exact-bytes claim; untangle the erro…
EricAndrechek Oct 1, 2026
e3c3f14
docs: a table whose schema cannot compile is unavailable on its own
EricAndrechek Oct 1, 2026
9d8f95a
docs: every copy of the unavailable claim names all four causes
EricAndrechek Oct 1, 2026
e07e0ce
fix(policy): resolve row-filter predicates in column order
EricAndrechek Oct 1, 2026
53d874e
refactor(api): inject the ClickHouse HTTP reader instead of a package…
EricAndrechek Oct 1, 2026
63d7f45
test(app): run the package's tests in parallel where they share no gl…
EricAndrechek Oct 1, 2026
741f086
Merge the internal/app test-time fix into chtypes-v2
EricAndrechek Oct 1, 2026
755c108
Merge origin/main (#711) into chtypes-v2
EricAndrechek Oct 1, 2026
2601574
fix(ingest): decline a body chtypes answers short, never report it short
EricAndrechek Oct 1, 2026
f0213ee
docs: correct the type layer's unavailable causes, NaN rendering and …
EricAndrechek Oct 1, 2026
90860b6
Merge branch 'chtypes-v2' into v2-uuid
EricAndrechek Oct 1, 2026
a318830
fix(ingest): read BOM- and form-feed-led bodies the way ClickHouse does
EricAndrechek Oct 1, 2026
24c4adc
feat(typelayer): move the type layer onto chtypes Go SDK v1.0.1
EricAndrechek Oct 4, 2026
d699bcf
docs(typelayer): name the unknown-zone refusal among the tenant-wide …
EricAndrechek Oct 4, 2026
0da30f7
docs: move the type-layer docs to chtypes 1.0
EricAndrechek Oct 4, 2026
0b51b43
build(chtypes): move the artifact fetch, lock, CI cache and images to…
EricAndrechek Oct 4, 2026
7b60408
Merge branch 'v1-docs' into chtypes-v2
EricAndrechek Oct 4, 2026
e6acc1f
docs: fix the chtypes 1.0 review findings
EricAndrechek Oct 4, 2026
2f14020
fix(typelayer): let only a servable tenant set the image zone; refuse…
EricAndrechek Oct 4, 2026
c224bcf
Merge branch 'v1-codefix' into chtypes-v2
EricAndrechek Oct 4, 2026
a0f2d81
Merge branch 'v1-docsfix' into chtypes-v2
EricAndrechek Oct 4, 2026
be86977
fix(typelayer): pin the artifact-error cause, scan records without oc…
EricAndrechek Oct 4, 2026
acd6a72
docs: document the image-zone latch and the remaining review findings
EricAndrechek Oct 4, 2026
513c1dc
docs: close the round-3 review findings on the image-zone latch and #456
EricAndrechek Oct 4, 2026
7fd3500
docs: tighten the image-zone and handle wording from the last review
EricAndrechek Oct 4, 2026
f1d2b3e
build(chtypes): move to chtypes go v1.0.2 and the 20261004.052404 art…
EricAndrechek Oct 6, 2026
ec1b3a6
fix(typelayer): drop the #458 latch from the setup-refusal cause
EricAndrechek Oct 6, 2026
522ad2a
docs: describe the type layer on chtypes 1.0.2
EricAndrechek Oct 6, 2026
3bf92db
docs(typelayer): cite the upstream issue for the remaining setup latch
EricAndrechek Oct 6, 2026
c36e7d2
test(typelayer): pin that a later tenant in a zone chtypes cannot loa…
EricAndrechek Oct 6, 2026
6d1dbdb
docs: scope the refused-zone cause to the first open and tighten the …
EricAndrechek Oct 6, 2026
30b9a07
docs: scope the later-tenant zone refusal by body format and name its…
EricAndrechek Oct 6, 2026
6a02ef9
feat(api)!: render NaN and infinities as strings on /v1/query and pipes
EricAndrechek Oct 6, 2026
0317182
feat(sdk)!: type generated Float columns as number or the nan/inf str…
EricAndrechek Oct 6, 2026
6484ce8
Merge origin/main (#729) into chtypes-v2
EricAndrechek Oct 6, 2026
c63011f
Merge branch 'v1-nanstrings' into chtypes-v2
EricAndrechek Oct 6, 2026
4e6809f
feat(typelayer): fetch a tenant's chtypes line on its first bind
EricAndrechek Oct 6, 2026
d502f1d
build: ship the chtypes CLI, not artifacts, and drop chtypes.lock
EricAndrechek Oct 6, 2026
187f76a
docs: describe autofetch, the chtypes cache, mirrors and the bundled CLI
EricAndrechek Oct 6, 2026
337010e
docs: say how a deployment refreshes its chtypes build, and cite #486…
EricAndrechek Oct 6, 2026
d3b14af
Merge branch 'v1-artifacts' into chtypes-v2
EricAndrechek Oct 6, 2026
67f6d2c
build(chtypes): move to chtypes go v1.0.3; a failed first open no lon…
EricAndrechek Oct 6, 2026
8ff492e
feat(ingest)!: refuse a body with a malformed record whole, as ClickH…
EricAndrechek Oct 6, 2026
8d3ac03
build(chtypes): move to chtypes go v1.0.4 and drop the wrong-line guard
EricAndrechek Oct 6, 2026
3afd3c1
Merge branch 'v1-sdk103' (chtypes go v1.0.4) into chtypes-v2
EricAndrechek Oct 6, 2026
e6c3e71
Merge branch 'v1-alignguard' (ClickHouse-native ingest) into chtypes-v2
EricAndrechek Oct 6, 2026
58e15be
fix(ingest): use the server's async inserts and wait for them, instea…
EricAndrechek Oct 6, 2026
ecceebb
docs: fix the round-7 review findings (artifact model, #468 leftovers…
EricAndrechek Oct 6, 2026
921b1bb
fix(ingest): isolate rejected rows with synchronous inserts
EricAndrechek Oct 6, 2026
b69a5cb
Merge branch 'v1-docsfix7' into chtypes-v2
EricAndrechek Oct 6, 2026
d569b0d
Merge branch 'v1-asyncinsert' into chtypes-v2
EricAndrechek Oct 6, 2026
fedcb20
docs: close the round-8 findings (CHECK exception, async-insert cause…
EricAndrechek Oct 6, 2026
226c9a4
build(chtypes): move to chtypes go v1.1.0 and use its strict cache mode
EricAndrechek Oct 6, 2026
0e5d733
fix(typelayer): narrow the missing-cache check and pin the cache-unus…
EricAndrechek Oct 6, 2026
da5e72b
docs: shared-handle scaling is fixed in the production chtypes build
EricAndrechek Oct 6, 2026
4339d2e
docs: close the v1.1.0 review findings (earlier-build figure, cache w…
EricAndrechek Oct 6, 2026
aafbce0
Merge origin/main into chtypes-v2
EricAndrechek Oct 6, 2026
04a3826
refactor(typelayer): take chtypes cache paths from the SDK and force …
EricAndrechek Oct 6, 2026
9d72d8b
fix(typelayer): pin cachePaths to the SDK and keep the autofetch pin …
EricAndrechek Oct 7, 2026
47cb96f
test(typelayer): keep TestCachePaths_MatchesTheSDK out of the real cache
EricAndrechek Oct 7, 2026
43be200
test(typelayer): fail TestCachePaths_MatchesTheSDK before it writes o…
EricAndrechek Oct 7, 2026
d06d30c
build(deployments): mount the chtypes cache at /var/cache/chtypes
EricAndrechek Oct 7, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/actionlint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,14 @@
# scope". Suppress only that exact message — a genuine scope typo (e.g.
# `contents` → `conten`) still produces a different message and fails. Drop
# this once actionlint ships the scope.
#
# nektos/act has the same gap and no equivalent escape hatch: `act` (0.2.89)
# refuses ci.yml outright with `Unknown Property code-quality` from its own
# schema validator, before running anything. That is the tool, not the
# workflow — GitHub accepts the scope. To dry-run or run ci.yml under act,
# copy the workflows to a scratch dir and strip the one `code-quality: write`
# line there. (act's default image also ships no Go, so `setup-env`'s
# `go version` step exits 127 under act regardless.)
paths:
"**/*.yml":
ignore:
Expand Down
73 changes: 73 additions & 0 deletions .github/actions/setup-env/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,13 @@
# lockfile + astro.config.mjs. Speeds up warm `astro check` /
# `astro build` — unchanged content skips the parse + transform
# pipeline. See #132.
# 7. chtypes artifact cache (`chtypes where`) keyed on the test
# ClickHouse line plus its newest build's digest — the .so the SDK
# dlopens. Measured for the 26.8 line (linux-amd64): ~340 MiB on disk
# (289 MiB unpacked .so, 49 MiB compressed layer). Every run fetches the
# line's newest build via scripts/fetch-chtypes.sh (two small requests
# when it is cached). Go test jobs that dial chtypes need this; see
# .github/workflows/README.md.
name: Setup CI environment
description: Caches (restore + automatic post-job save) and toolchains for WaveHouse CI

Expand All @@ -66,6 +73,9 @@ inputs:
astro:
description: "Cache the Astro content collections (docs check/build)"
default: "false"
chtypes:
description: "Fetch + cache the chtypes artifact for the test ClickHouse line (Go test jobs that dial chtypes only)"
default: "false"

# Exact-key cache-hit flags, for consumers that want to skip work on a
# warm cache (e.g. docs-build's pnpm-store prune). Saves are NOT gated on
Expand Down Expand Up @@ -178,6 +188,65 @@ runs:
restore-keys: |
golangci-${{ runner.os }}-

# chtypes artifact cache (the v1 OCI layout). There is no lock: the key is
# the test ClickHouse line (from internal/chversion) plus the digest of
# that line's newest build for this platform, read from the registry's
# index (one ~1 KB request, unverified: it only names the entry). A new
# upstream build is a new key, so the post-job save captures it; the
# restore-keys bring back the previous build, the fetch below installs the
# new one on top, and --prune drops the old before the save, so an entry
# holds one build. runner.arch is in the key: an arm64 runner must never
# restore amd64 .so files. The path is `chtypes where`, so the SDK's layout
# version is not named here; the key's `v2` is this cache's shape (v1 was
# keyed on the retired lock).
- name: Resolve the chtypes artifact cache key
if: ${{ inputs.chtypes == 'true' }}
id: chtypes-key
shell: bash
run: |
case "$RUNNER_OS" in Linux) os=linux ;; macOS) os=darwin ;; *) os=unknown ;; esac
case "$RUNNER_ARCH" in X64) arch=amd64 ;; ARM64) arch=arm64 ;; *) arch=unknown ;; esac
echo "line=$(scripts/fetch-chtypes.sh --print-line)" >> "$GITHUB_OUTPUT"
cache="$(scripts/fetch-chtypes.sh --where)"
echo "cache=$cache" >> "$GITHUB_OUTPUT"
if digest="$(scripts/fetch-chtypes.sh --resolve --platform "$os-$arch")"; then
echo "build=$(printf '%s' "${digest#sha256:}" | cut -c1-16)" >> "$GITHUB_OUTPUT"
else
echo "::warning title=chtypes::could not resolve the newest build; restoring the last cached one without saving"
fi

- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
if: ${{ inputs.chtypes == 'true' && steps.chtypes-key.outputs.build != '' }}
id: chtypes-cache
with:
path: ${{ steps.chtypes-key.outputs.cache }}
key: chtypes-v2-${{ runner.os }}-${{ runner.arch }}-${{ steps.chtypes-key.outputs.line }}-${{ steps.chtypes-key.outputs.build }}
restore-keys: |
chtypes-v2-${{ runner.os }}-${{ runner.arch }}-${{ steps.chtypes-key.outputs.line }}-

# Registry unreachable: restore the line's newest entry, and save nothing
# under a key that names no build.
- uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
if: ${{ inputs.chtypes == 'true' && steps.chtypes-key.outputs.build == '' }}
with:
path: ${{ steps.chtypes-key.outputs.cache }}
key: chtypes-v2-${{ runner.os }}-${{ runner.arch }}-${{ steps.chtypes-key.outputs.line }}-
restore-keys: |
chtypes-v2-${{ runner.os }}-${{ runner.arch }}-${{ steps.chtypes-key.outputs.line }}-

# Every run resolves the line's newest build and installs it unless it is
# cached: two small requests on a warm cache, the ~45 MB layer otherwise.
# chtypes verifies the signed statement on every install. A registry
# outage falls back to the cached build, loudly, rather than failing CI.
- name: Fetch the chtypes artifact
if: ${{ inputs.chtypes == 'true' }}
shell: bash
run: |
if ! scripts/fetch-chtypes.sh --prune; then
echo "::warning title=chtypes::fetching the newest build failed; testing on the cached build"
scripts/fetch-chtypes.sh --offline
fi

# No actions/setup-go: it spends ~9s/job downloading + extracting a
# toolchain the runner can already provide. The image's preinstalled
# `go` + GOTOOLCHAIN=auto (the Go ≥1.21 default) resolve go.mod's
Expand All @@ -189,6 +258,10 @@ runs:
# this step makes that cost visible and the post-job save captures it.
# Trade-off: setup-go's inline problem matchers (PR-file annotations
# on compile errors) are gone; the log output is unchanged.
#
# On a chtypes job the toolchain fetch happens earlier, inside the chtypes
# steps above: scripts/fetch-chtypes.sh runs the CLI through this repo's
# go.mod, so it resolves the same toolchain this step reports.
- name: Verify Go resolves go.mod's toolchain
if: ${{ inputs.go == 'true' }}
shell: bash
Expand Down
7 changes: 7 additions & 0 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,13 @@
- any-glob-to-any-file:
- "internal/pipes/**"

"area/typelayer":
- changed-files:
- any-glob-to-any-file:
- "internal/typelayer/**"
- "internal/chversion/**"
- "scripts/fetch-chtypes.sh"

"area/sdk":
- changed-files:
- any-glob-to-any-file:
Expand Down
Loading
Loading