Skip to content

ci: verify the SQLite JDBC download before installing it into Lucee - #4594

Merged
bpamiri merged 2 commits into
developfrom
peter/ci-sqlite-jdbc-download-check
Oct 14, 2026
Merged

bpamiri merged 2 commits into
developfrom
peter/ci-sqlite-jdbc-download-check

Conversation

@bpamiri

@bpamiri bpamiri commented Oct 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

CI's SQLite JDBC download used curl -sL, which exits 0 on an HTTP error page or a truncated transfer. A damaged jar then failed much later and less clearly: Tomcat logged zip END header not found for lib/ext/sqlite-jdbc.jar, and the core suite errored with org.sqlite.JDBC not found.

The download now runs through tools/ci/fetch-sqlite-jdbc.sh in all three places that fetched it: pr.yml, snapshot.yml and the setup-wheels-test-env action. The script:

  • downloads with curl --fail, retrying on any error;
  • checks the jar against Maven Central's published SHA-256 for sqlite-jdbc 3.50.3.0;
  • on a mismatch, downloads again, up to three attempts;
  • otherwise fails the step with an ::error::, the URL, and the hash and size it actually got.

The Docker test images download the same jar with wget, which already exits non-zero on HTTP errors. They're unchanged here.

Verification

Case Result
Real download Verified, exit 0. unzip -t reports no errors.
Pinned hash altered Three attempts, each reporting the hash and size it got; then ::error:: and exit 1. No output file is left behind.
URL returning 404 curl reports the 404 on each attempt; then ::error:: and exit 1.

shellcheck passes on the script. actionlint on pr.yml and snapshot.yml reports the same 10 findings as on develop, all from existing steps.

🤖 Generated with Claude Code

The PR, snapshot and shared test-env setup steps fetched sqlite-jdbc with
`curl -sL`, which exits 0 on an HTTP error page or a truncated transfer. A
damaged jar then failed much later: Tomcat logged "zip END header not
found" and the core suite errored with org.sqlite.JDBC not found.

All three now run tools/ci/fetch-sqlite-jdbc.sh. It downloads with
`curl --fail` and retries, checks the jar against Maven Central's
published SHA-256, downloads again on a mismatch (three attempts), and
otherwise fails the step with the URL, the hash and the size it got.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Peter Amiri <peter@alurium.com>
@bpamiri

bpamiri commented Oct 14, 2026

Copy link
Copy Markdown
Collaborator Author

rev1-r3 review @ 0c1d652: CHANGES (one CI failure caused by this PR; the script itself is good)

Blocking: Lucee 7 + SQLite (LuCLI) fails on a version-pin guard this PR breaks

Failed: wheels start stages 3.49.1.0; test-cli-local.sh downloads 3.49.1.0; CI pr.yml uses 3.50.3.0:
The needle [sqlite-jdbc/3.50.3.0/sqlite-jdbc-3.50.3.0.jar] was not found in [name: Wheels Pull Req…

cli/lucli/tests/specs/commands/CliHardenerS1S10Spec.cfc § "S9 HOLD — SQLite JDBC pins stay forked" reads pr.yml and expects the 3.50.3.0 URL there. That URL now lives in tools/ci/fetch-sqlite-jdbc.sh. The hold's intent still holds (CI uses 3.50.3.0, the CLI paths 3.49.1.0), so update the guard rather than drop it:

  • assert pr.yml runs tools/ci/fetch-sqlite-jdbc.sh;
  • assert the script carries sqlite-jdbc/${VERSION}/… with VERSION="3.50.3.0" and no 3.49.1.0.

The script is good

  • The expected hash is pinned in the repo (SHA256="a3f53a2a…cf08"), not fetched at runtime, so it catches a swapped jar as well as a damaged one. I checked the value: it equals Maven Central's published sqlite-jdbc-3.50.3.0.jar.sha256 and the SHA-256 of the jar I downloaded, and the published .sha1 (220f0cdf…) matches too.
  • No partial file. Each attempt downloads to ${OUT}.download. The output path is written only by mv after the hash matches, and .download is removed before each attempt and on final failure. curl -fsSL --retry 3 --retry-all-errors fails on HTTP errors. The [ "$attempt" -lt 3 ] && sleep 5 tail is errexit-safe.

Not every CI download goes through it yet

The PR body says "all three places that fetched it". Two more CI paths fetch sqlite-jdbc without a hash:

  • deploy-ci.yml → tools/test-cli-local.sh (L274): curl -sL …/3.49.1.0/sqlite-jdbc-3.49.1.0.jar straight into Express lib/ext. This is the same "exits 0 on an error page or truncation" failure, on a GitHub runner.
    • Because of the S9 hold it must stay at 3.49.1.0. So either let the helper take a version and hash (the 3.49.1.0 SHA-256 is already pinned in build-linux-packages.sh: 5c8609d2…9291), or add the same pinned-hash check there.
    • tools/test-cli-e2e.sh has the same pattern, but no workflow runs it.
  • tools/docker/lucee{5,6,7}/Dockerfile (compat matrix, cold-start, compat-pr): wget fails on HTTP errors, as the PR notes, but checks no hash. A sha256sum -c after each wget would close it.

Both can be follow-ups if you'd rather keep this PR small. The guard fix above is needed here for CI to go green.

Other checks: 15 pass, 3 skipping. The new step logs sqlite-jdbc 3.50.3.0 downloaded and verified. Up to date with develop, MERGEABLE.

tools/ci/fetch-sqlite-jdbc.sh takes an optional version and SHA-256, so the
3.49.1.0 driver the CLI stages is now downloaded through it as well:
test-cli-local.sh (the deploy-ci CLI suite), test-cli-e2e.sh and
test-local.sh pin it by the SHA-256 the release packages already use. No
`curl -sL` download of the driver remains.

CliHardenerS1S10Spec's S9 hold read the 3.50.3.0 URL from pr.yml, which now
lives in the helper. It now checks that pr.yml calls the helper with its
default, that the helper defaults to 3.50.3.0, and that test-cli-local.sh
passes 3.49.1.0 with a SHA-256.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Peter Amiri <peter@alurium.com>
@bpamiri

bpamiri commented Oct 14, 2026

Copy link
Copy Markdown
Collaborator Author

rev1-r3 re-check @ 9154b4f: APPROVE

Both points from my CHANGES at 0c1d652 are resolved.

1. The S9 HOLD guard is updated, not dropped. CliHardenerS1S10Spec now asserts:

  • pr.yml runs bash tools/ci/fetch-sqlite-jdbc.sh /tmp/sqlite-jdbc.jar (the default version) and has no 3.49.1.0;
  • the helper defaults to VERSION="3.50.3.0" and has no 3.49.1.0;
  • test-cli-local.sh calls the helper with 3.49.1.0 <64-hex> and has no 3.50.3.0;
  • the module still stages only 3.49.1.0.

The hold's intent, CLI on 3.49.1.0 and CI on 3.50.3.0, is still pinned. The suite passes on this head.

2. Every CI fetch is verified.

  • The helper takes an optional version sha256 pair. It must be both or neither (anything else is exit 2), and the default stays the pinned 3.50.3.0.
  • test-cli-local.sh (run by deploy-ci.yml), test-cli-e2e.sh and test-local.sh now fetch 3.49.1.0 through it, pinned to 5c8609d2…9291, the same hash build-linux-packages.sh uses.
  • I checked that hash independently. Maven Central publishes no .sha256 for 3.49.1.0, but a fresh download hashes to 5c8609d2…9291, and its SHA-1 equals the published .sha1 (145d3a68…).
  • No curl -sL driver download remains in CI paths.
  • The Docker wgets are tracked in Verify the SQLite JDBC jar's SHA-256 in the Lucee test images #4595.

CI on this head: 19 pass, 2 skipping, 0 fail. Both download paths are exercised and verified in the logs:

  • Lucee 7 + SQLite (LuCLI): sqlite-jdbc 3.50.3.0 downloaded and verified (sha256 a3f53a2a…), then the CLI suite at 2449 passed, 0 failed, 0 errors;
  • Deploy CLI tests + verb smoke: sqlite-jdbc 3.49.1.0 downloaded and verified (sha256 5c8609d2…), then 2449 pass, 0 fail, 0 error.

MERGEABLE.

INFO, not blocking: the local onboarding harness (tools/test-onboarding.sh L528) still downloads 3.49.1.0 with curl -fsSL and no hash. It fails on HTTP errors and isn't run in CI. It could use the helper the same way when convenient.

@bpamiri
bpamiri merged commit f030f38 into develop Oct 14, 2026
21 checks passed
@bpamiri
bpamiri deleted the peter/ci-sqlite-jdbc-download-check branch October 14, 2026 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant