Repository navigation
fork: outside PRs run nothing, one checkout without actions/checkout, contract tests - #12
Conversation
… contract tests
- fork-tests.yml, fork-checks.yml: every job runs only for a pull request whose head is in this
repository (no pull_request_target run for outside forks).
- fork-release.yml, fork-follow.yml: the eight actions/checkout steps (persist-credentials: false
fails on upstream's gitlinks without .gitmodules) are one shared plain-git step: anonymous
fetch of one ref, full (blob:none) or tip history, optional sparse set, fails closed.
- fork-contracts.test.py: the pull request guard on every PR-started job and an allowlist of
fork triggers; the shared checkout run end to end on a repository with a gitlink and no
.gitmodules; no refusal in fork-follow.yml swallowed (|| true, || :, ; true, set +e, trap,
continue-on-error, if:, another shell, BASH_ENV); every script calling gh sources
fork-gh-guard.sh, every shell test sets up the gh stub first. Each rule with broken copies.
- fork-schema-check.test.cjs: whole run: | blocks are scanned for ${{ }}, github.token banned.
- The three shell tests set up the gh stub before anything else; fork-secret-scan.test.sh gets
one that refuses every call.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
lens: security+correctness S-1 · reviewer: opus high (t3lens-12-s1) · read at: 18dadd6
|
|
lens: security+correctness S-2 · reviewer: opus high (t3lens-12-s2) · read at: 18dadd6
|
…fetch)
- An outside pull request now fails: a new outside_pr job (fork-tests.yml and
fork-checks.yml) checks nothing out, has permissions {} and exits 1.
fork_ci_result runs with always(), needs outside_pr and refuses an outside
pull request first. A job skipped by its if: counts as passed, so before this
a required fork-ci-result went green on an untested outside pull request.
- fork-contracts.test.py, refusals: a refusal step may hold only the report
folder and its script call (a whitelist). Earlier steps in its job may be only
the shared checkout, another refusal or one step-output line. Nothing may write
GITHUB_ENV, GITHUB_PATH or BASH_ENV, define a function or use an action. No
BASH_FUNC_* env.
- gh-guard: backslash continuations are joined before matching. The guard line
must stand at the top level, after nothing but #!, set and comments.
- fork-schema-check.test.cjs: runScripts reads `run:` alone with the script on
the next line, and plain scalars continued over later lines.
- The shared checkout fetches refs/heads/<REF> unless REF is a 40-hex SHA, so a
tag named like the branch is never taken. e2e: a tag `main` on an older commit.
- Comments: fork-tests.yml says the guard stops only honest pull requests (the
PR's own copy runs under pull_request; the approval setting protects). The
shared step's comment no longer claims fork-tests.yml uses it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
not a lens: fix round r2 · head e76f0a3 (was 18dadd6) This answers the findings in S-1 (#issuecomment-6826888015) and S-2 (#issuecomment-6826894827). The PR body has a new section, "Fix round r2", with its state-question table and a "Not done" list.
Tests at e76f0a3, in a scratch clone, with GH_TOKEN/GITHUB_TOKEN unset and GH_HOST=bogus.invalid: fork-contracts.test.py (89 ok, all passed), fork-release-workflow.test.py, fork-follow.test.sh, fork-release.test.sh, fork-secret-scan.test.sh all pass. fork-schema-check.test.cjs 51/51, fork-ci-tests.test.cjs 8/8, fork-test-shards.test.cjs 4/4, fork-tests-drift.test.cjs 6/6, and fork-tests-drift.cjs: "match". actionlint 1.7.12 with shellcheck 0.10.0 is clean on fork-*.yml. oxfmt is clean on the changed .cjs and .yml. Not done (owner's call, in the PR body): deleting thread-transfer-report.yml and mobile-fingerprint-label.yml. |
|
lens: correctness+security r3 · reviewer: opus high (t3lens-12-r3) · read at: e76f0a3
|
Conflicts in fork-tests.yml: each routed job keeps #13's needs and runs-on and takes main's pull request guard in its if:. The new `route` job gets the guard too (fork-contracts.test.py: every job a pull request starts begins with it). fork-ci-tests.txt lists both new tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follow-up to #3, #4 and #8: small leftovers from their reviews, plus one owner request.
What changes
fork-tests.ymlandfork-checks.ymlnow starts with
(!github.event.pull_request || github.event.pull_request.head.repo.full_name == github.repository). A deleted headrepository reads as null, so the job doesn't run. The exception (fix round r2) is
outside_pr, which runs only foran outside PR, checks nothing out, has
permissions: {}and exits 1. Infork-tests.yml,fork_ci_resultruns too,needs it and fails, so a required check never goes green on an untested outside PR. In
fork-checks.yml(
pull_request_target, always main's copy), the guard is enforced by main. Infork-tests.yml(pull_request, thePR's own copy), an outside PR can delete the guard, and its code runs once a maintainer approves the run. There the
approval setting is the real protection, so it must stay on, and outside runs of fork-tests must never be approved.
fork-schema-check.test.cjsnow checks wholerun: |blocks for${{ }}, not only therun:line.github.token.actions/checkoutsteps infork-release.yml, and the four with the same bug infork-follow.yml, are replaced by one shared plain-git step. It fetches one ref anonymously with--no-recurse-submodules, with full history (blob:none) or only the tip, plus an optional no-cone sparse set,then detaches. It fails on an empty REF, an unknown ref, a bad HISTORY or a workspace that already has a checkout.
.git/configholds no credential.fork-follow.test.shlooked only for continue-on-error. It is replaced by one thatcatches every way to swallow a refusal:
|| true,|| :,|| exit 0,; true, a trailingtrue,&,!,if …; then,set +e,set +o errexit, a trap, continue-on-error on the step or job, anif:on a refusal step,another shell or
defaults, and BASH_ENV at workflow, job or step level.fork-gh-guard.shat the top level before its firstcall. A non-shell script that calls gh fails the check. Every
fork-*.test.shmust callfork_test_ghbeforeanything except set-up lines, and a non-shell test may not spawn gh or a guarded script.
fork-secret-scan.test.shgets a stub that refuses every call.fork-follow.test.shnow builds its refusal list from the scripts that source the guard, sofork-release.shis included.
The new
.github/scripts/fork-contracts.test.pyholds items 1, 4, 5 and 6. Each rule runs on the real files and onbroken copies that must fail.
Red before, green after
The new tests run against main's files in a scratch copy, or against main plus one mutation:
pr-guard: every job a pull request can start …FAIL (every job of both files)${{ github.event.pull_request.title }}inside arun:blockthe workflow runs … never runs pull request codeFAIL on it; the mutant test is okGH_TOKEN: ${{ github.token }}in an envcheckout: no actions/checkout drops credentials …and… preflight, build_bundle, assemble and publish use itFAIL; main's fork-follow.yml: the same for guard, plan, advance and reportfork-follow.test.sh"all passed" with|| trueon the guard steprefusals: no refusal … can be swallowedFAILgh-guard: every fork shell test sets up the gh stub …FAIL;fork-workflows.shwithout its guard line:… sources fork-gh-guard.sh firstFAILAll fork tests pass locally with GH_TOKEN and GITHUB_TOKEN unset and GH_HOST=bogus.invalid. actionlint 1.7.12, the
CI's version, run with shellcheck, reports nothing. shellcheck adds no warning compared with main. oxfmt (vite-plus
1.4.0) is clean.
Checkout against the real fork: I ran the shared step anonymously (empty git config, no credential helper, no
token) against
https://github.com/scuttletoby/t3code, main 8b0ad2c:/* !/.repos/).reposon disk/.github/scripts/).github/All three ran from that main, which has 2 gitlinks and no
.gitmodules.git submodule foreach(what actions/checkout'sclean-up runs) fails on that checkout with
No url found for submodule path '.repos/alchemy-effect/submodules/distilled'.No release was made and nothing was published.
What happens when this runs again, stops half-way, or meets existing, old or empty state?
checkout e2e fork-*.yml/<job>(each job's own values)checkout e2e: a workspace that already holds a checkout failscheckout e2e: an empty REF fails the stepcheckout e2e: an unknown SHA / an unknown branch fails the step.gitmodulesgit submodule foreachfails on the same fixturecheckout e2e: the fixture breaks git submodule foreach …checkout e2e: full history reaches the first commitcheckout e2e: an unknown HISTORY fails the stephead.repois null, so it counts as an outside PR. The guarded jobs don't run, andoutside_prandfork_ci_resultfail (r2)pr-guard: the guard compares the head repository …,outside_pr's if is the guard's exact negation… failsline in fork-contracts.test.py and the schema-check mutant testItem 1: what the sweep can and can't disable
Workflows that are registered and could be started from outside are all
disabled_manually:pull_request_target: pr-limits, pr-size, pr-vouch, relay-preview-cleanup, desktop-macos-preview-publish.pull_request: ci, web-preview, relay-preview, the mobile ones, agent-context-budget.Upstream workflows on main that are not registered can't be disabled:
GET actions/workflows/<file>returns 404. #4'ssweep (
fork-workflows.sh) runs only from fork-follow, daily or on dispatch, so it can disable a file only on the runafter GitHub registers it. That first run is not stopped. The ones an outside event could start:
mobile-eas-preview-fork.ymlpull_request_target: labeleddesktop-macos-preview.ymlpull_request: labeledpreview:mac;pull_requestalso waits for approval; needs a blacksmith runnermobile-fingerprint-label.ymlworkflow_runof "Mobile Fingerprint Check"thread-transfer-report.ymlworkflow_runof "CI"issue-labels.ymlruns on push and dispatch only, and Issues are off on this fork. If the owner wants this closedrather than relying on labels and runners, the fork patch could delete
mobile-eas-preview-fork.ymlanddesktop-macos-preview.yml. This PR doesn't do that.Fix round r2 (after the two security lenses)
outside_pr, infork-tests.ymlandfork-checks.yml. Itsif is the guard's exact negation. It checks nothing out, has
permissions: {}, gets no secrets, has one step andexits 1 with
outside PR: not tested here; a maintainer must review.fork_ci_resultnow runs withalways()andneeds
outside_pr. Its first lines refuse whenOUTSIDE_PRis true, and its jq letsoutside_prpass only asskipped. The contract test holds that shape exactly, with 12 mutants. It also runs
fork_ci_result's script on fiveresult sets: outside PR with the rest skipped, outside PR with the rest passed,
outside_prfailed on an own PR,own PR all passed, and a push.
may have only
id,name,envandrun. Its run may hold onlymkdir -p "$RUNNER_TEMP/reports"and one scriptcall. Before it in its job may stand only the shared checkout, another refusal, or one
echo "x=…" >> "$GITHUB_OUTPUT"line. No earlier step may use an action, write GITHUB_ENV, GITHUB_PATH or BASH_ENV, or define a function. No
BASH_FUNC_*env at any level. The old blacklist stays only for the other script steps. In gh-guard, the guard linemust stand at the top level, after nothing but
#!,setand comments.${{ }}in run scripts.runScriptsnow takes every line indented past therun:key, whatever the valuelooks like. That covers
run:alone with the script on the next line, and a plain scalar continued over laterlines.
\are joined before matching.refs/heads/<REF>unless REF is a 40-hex SHA. A tag named like the branch is never taken, and arefs/tags/…REF fails. e2e: the origin has a tagmainon its older commit.fork-tests.ymlnow says the guard stops only honest PRs and the approval setting is the protection.The shared step no longer says every fork job uses it:
fork-tests.ymlkeeps its 12actions/checkoutsteps.Moving them isn't small. Most have an empty
refthat falls back to the event's commit, and they usefetch-depth: 0and2and their own sparse sets. They follow upstream'sci.ymlstep for step, which the driftcheck compares.
r2: what happens when this runs again, stops half-way, or meets existing, old or empty state?
outside_prfails again andfork_ci_resultfails again, on every attemptfork_ci_result's script: an outside pull request …outside_prstays skipped.fork_ci_resultre-runs and sees the same results… own-branch pull request with every job passed passesOUTSIDE_PRis empty andoutside_pris skipped.fork_ci_resultworks as before… a push (OUTSIDE_PR empty) … passesfork_ci_result(always()) seescancelledresults and fails, as beforerefs/tags/…REF fails the stepcheckout e2e: a branch name takes the branch …,a tag's full ref fails the steprefs/heads/<name>is missing, so the step fails (as before)checkout e2e: an unknown branch fails the stepgit remote add, as beforecheckout e2e: a workspace that already holds a checkout failsNot done (owner's call)
thread-transfer-report.ymlandmobile-fingerprint-label.ymlin the fork patch. Both start onworkflow_runof a workflow name ("CI", "Mobile Fingerprint Check"). An outside PR can add a workflow with thatname, and once its run is approved, its completion starts these from main with
pull-requests: write. They read theartifact as data, so the impact is limited to a PR comment's text.
Notes for the owner
fork-tests.ymlisdisabled_manuallysince 2026-11-12T14:12:38Z, about a minute after main's push run43234574679. No Actions run explains it. This PR doesn't change it, because repo settings are off limits, but its
Fork tests won't run until the workflow is enabled again.
fork_ci_result, and a skipped job counts as passing for a required check.Now
outside_prandfork_ci_resultfail on it. Forfork-checks.yml, a ruleset that requires "No database schemachanges" must also require "Outside pull request" (it is skipped, so green, on our own branches).
fork-checks-proof.yml(the proof workflow from fork: green main again (schema-check comments, checkout without submodule cleanup) #8) is still registered and active.🤖 Generated with Claude Code