Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/agentic-fuzzing/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ routed to its own line's executor and silently runs another one).
Lower it to `safe` when you need to be sure a divergence is the contract's and
not a debug facility's — `safe` refuses both wall-clock exposure and the `:test`
alias. Once `:test` no longer resolves the contract must name its runner by
hash, read out of `build/out/executor/<version>/data/latest.json`, e.g.
hash, the runner's `latest` in `build/out/executor/<version>/data/all.json`, e.g.
`# { "Depends": "py-genlayer:9b8kjy…" }`. That hash changes whenever the SDK
does, which is exactly why in-tree cases keep the alias and only probes give it
up.
Expand Down
53 changes: 53 additions & 0 deletions .github/workflows/audit_nightly.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: audit / nightly

# Publishes `genvm-tool audit` findings to code scanning (Security → Code
# scanning, tool "genvm-tool audit"). An alert closes by itself once a run no
# longer reports it; dismissing one there is the ruling that it does not apply.
#
# GitHub only schedules workflows from the DEFAULT branch, so this file has to
# reach it before the cron fires.

on:
schedule:
# Off the hour: a scheduled run on a busy minute is queued or dropped.
- cron: "37 4 * * *"
workflow_dispatch:

permissions:
contents: read
security-events: write

concurrency:
group: audit-nightly
cancel-in-progress: false

defaults:
run:
shell: bash -exo pipefail {0}

jobs:
audit:
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
# Lockfiles are tracked in the submodules; no third-party tree is read
- name: Get source
uses: ./.github/actions/get-src
with:
with_nix: "true"
third_party: none
github_token: ${{ secrets.GITHUB_TOKEN }}
nix_cache_pull_token: ${{ secrets.NIX_CACHE_PULL_TOKEN }}
- name: Audit
env:
# Optional: without it NVD is queried at its anonymous rate
NVD_API_KEY: ${{ secrets.NVD_API_KEY }}
run: >-
nix develop '.?submodules=1#audit' --command
genvm-tool audit --sarif build/audit.sarif
- name: Publish to code scanning
uses: github/codeql-action/upload-sarif@1fc58d04a4d4374f921bdac573c3811c240f9c26 # v4.39.2
with:
sarif_file: build/audit.sarif
category: genvm-tool-audit
61 changes: 0 additions & 61 deletions .github/workflows/branch_wasmtime_watch.yaml

This file was deleted.

26 changes: 7 additions & 19 deletions docs/contributing/howto/committing/git-third-party.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,30 +59,18 @@ the nested repo's commits and `save`

## Advisory Monitoring

`branch / wasmtime watch` checks every vendored source of every active line
against [OSV](https://osv.dev) daily and keeps one `wasmtime-maintenance` issue
in sync with the result. It gates nothing and writes no branch. Run it by hand
with `./support/ci/run.sh tool wasmtime-watch --dry-run`

Two things are queried per line, because neither covers the other:
`audit / nightly` runs `genvm-tool audit` daily and publishes to Security →
Code scanning ([genvm-tool.md](../genvm-tool.md)). Vendored sources are checked
twice, because neither check covers the other:

1. each repo in `manifest.json`, by its pinned upstream commit
2. each crate that comes out of one, by its locked version — a vendored tree is
dozens of crates (`cranelift-codegen`, `wasmparser`, `wiggle`), and that is
where most advisories against this stack are actually published

A crate counts as vendored when `executor/Cargo.lock` gives it no `source` and
no tracked `Cargo.toml` in the executor repo declares it. Registry crates are
excluded because they are covered upstream; first-party crates, because they are
ours. A first-party crate with no committed manifest falls through and is
queried too — a dismissible false positive, chosen over a silent gap
A crate counts as vendored when `Cargo.lock` gives it no `source` and no tracked
`Cargo.toml` declares it

OSV knows nothing about the features GenVM disables or what the patch series
changes, so a hit is a finding to rule on, not a proven exposure

The sweep edits its issue in place and never closes or reopens one: closing it
is the ruling that its findings are handled. The advisory ids that issue covered
are recorded in its body marker, so later sweeps stay quiet until an id outside
that set appears, at which point a fresh issue supersedes it.
`WASMTIME_REBASE_OWNER` (an Actions variable) is the login the issue is assigned
to at creation; sweeps never reassign it
changes, so an alert is a finding to rule on, not a proven exposure. Dismissing
it in code scanning is that ruling
15 changes: 15 additions & 0 deletions docs/contributing/howto/extending/modify-runner.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,18 @@ Paths are relative to `executors/v0.3.x/`, the line that owns runner sources

Step 4 in detail, and why a hash is never restored by hand:
[committing/runners.md](../committing/runners.md)

## Gating a Runner

A pin in `current.nix` may set 2 optional attributes, which end up in the
line's `data/all.json`:

1. `available-after = "2026-12-01T00:00:00Z";` — transactions timestamped
earlier resolve the runner as malformed, the same as on an executor that does
not ship it ([spec](../../../website/src/spec/02-execution-environment/04-runners.rst)).
Pick a time by which every validator runs the new executor. A runner new to
a released minor must carry one ([versioning](../../../website/src/spec/01-core-architecture/03-versioning.rst))
2. `supersedes = "<gvm32 hash>";` — the hash of the same id this one replaces.
The old hash must still be shipped, or evaluation fails

`hash-updater.py` rewrites only `hash`, so both survive a re-pin
1 change: 1 addition & 0 deletions docs/contributing/howto/genvm-tool.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ has no wrapper
| `codegen` | render a data JSON into rust/python/rst/go |
| `build-manifest` | write the manager's `data/manifest.yaml` |
| `docs` | render the `docs/contributing/` indexes; `--write` splices them into `AGENTS.md` (`CLAUDE.md` is a symlink to it), `--check` fails when that copy is stale |
| `audit` | list published advisories against tracked lockfiles (osv-scanner), vendored trees by pinned commit (OSV) and the `cpes` of `libs/deps/dependency-urls.json` entries (NVD); `--markdown` writes a report, `--sarif` feeds the nightly `audit / nightly` upload to Security → Code scanning |
| `fuzz-corpus` | seed the fuzz corpora from a finished run ([testing/fuzzing.md](testing/fuzzing.md)) |
| `git` | `ls`, `list-repo`, `create-branches`, `check-for-push` across the manager and every executor submodule ([submodules.md](committing/submodules.md)) |

Expand Down
2 changes: 1 addition & 1 deletion docs/contributing/howto/testing/integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ because `reroute_to` — what points a case at its own line's executor — is
honored only from `safe` up. Below `unsafe` a contract must name its runner by
hash instead of `:test`. Write `@RUNNER_LATEST_<line>_<runner>@` for that hash,
for example `py-genlayer:@RUNNER_LATEST_v0.3_py-genlayer@`: the harness replaces
it with the uid in `build/out/executor/<version>/data/latest.json` before the
it with the runner's `latest` in `build/out/executor/<version>/data/all.json` before the
code reaches an executor, so a runner change needs no test edit. A literal uid
still works and goes stale

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,10 @@ Minor
A minor release adds new features and is backwards-compatible.

#. The :ref:`gvm-def-execution-hash` of a transaction may change.
#. New :term:`Runner`\s may be added; existing runners MUST NOT be removed.
#. New :term:`Runner`\s may be added; each MUST carry an
:ref:`availability time <gvm-def-runner-availability>`, so executors of the
previous minor agree with it until then. Existing runners MUST NOT be
removed.
#. New ``gl_call`` methods and new parameters of existing methods may be added.
#. Old code continues to work, unless it deliberately tried to be
future-incompatible (e.g. by observing additions permitted by rule 3).
Expand Down
14 changes: 14 additions & 0 deletions docs/website/src/spec/02-execution-environment/04-runners.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,20 @@ A ``chain:`` id is code, but its resolution is an ordinary
A resolution that finds no code, or code that is not a valid :term:`runner`,
fails like any other malformed runner rather than falling back to another view.

.. _gvm-def-runner-availability:

Packaged Runner Availability
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

An executor ships a fixed set of packaged runners. Each may carry an
availability time:

#. A ``<human-readable-id>:<hash>`` the executor does not ship fails as a
malformed runner
#. One whose availability time is later than the transaction timestamp fails
the same way, so executors that ship it agree with those that do not until
that time

Hash Format
~~~~~~~~~~~

Expand Down
3 changes: 3 additions & 0 deletions docs/website/src/spec/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -134,3 +134,6 @@ Changed
:term:`Storage Slot` length instead of by ``u32`` overflow, so a slot's final
octet is addressable; every access naming it was previously refused. See
:doc:`02-execution-environment/03-wasi_genlayer_sdk/01-functions`
#. A packaged runner may carry an availability time; a transaction timestamped
before it resolves the runner as malformed, as if it were not shipped. See
:ref:`gvm-def-runner-availability`
7 changes: 7 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,8 @@
python312Packages.pytest-cov
python312Packages.pytest-xdist
];
# The scanner `genvm-tool audit` drives
packages-audit = [ pkgs.osv-scanner ];
shell-hook-base = ''
export CARGO_LD_LIBRARY_PATH="${toString pkgs.xz.out}/lib:${toString pkgs.zlib.out}/lib:${pkgs.stdenv.cc.cc.lib}/lib:${toString pkgs.glibc}/lib"
export LLVM_PROFILE_FILE=/dev/null
Expand Down Expand Up @@ -776,6 +778,7 @@
++ packages-py-test
++ packages-rust
++ packages-gen-docs
++ packages-audit
++ [ pkgs.nodejs ]
# pre-commit + the hook tools, so `pre-commit run` works in-shell.
# (No rustfmt/cargo here — cargo-fmt is a custom hook — so the
Expand All @@ -787,6 +790,10 @@
# by pre-commit's supported `.legacy` hook mechanism.
shellHook = shell-hook-base + pre-commit-check.shellHook + install-manager-push-refs;
};
devShells.audit = pkgs.mkShell {
packages = packages-0 ++ packages-audit;
shellHook = shell-hook-base;
};
devShells.check-qemu = pkgs.mkShell {
packages = packages-0 ++ [ pkgs.qemu ];
shellHook = shell-hook-base;
Expand Down
13 changes: 7 additions & 6 deletions install/lib/python/post-install/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -382,13 +382,14 @@ def _load_registry(file: str | Path) -> dict[str, list[str]]:

ret: dict[str, list[str]] = {}

# Frozen legacy lines ship `{id: [hash]}`, the others `{id: {versions: {hash: ...}}}`.
for k, v in sorted(contents.items()):
if isinstance(v, str):
ret[k] = [v]
elif isinstance(v, list):
if not all([isinstance(x, str) for x in v]):
raise RuntimeError(f'registry value must be str | list[str] for {k}')
if isinstance(v, dict) and isinstance(v.get('versions'), dict):
ret[k] = list(v['versions'])
elif isinstance(v, list) and all(isinstance(x, str) for x in v):
ret[k] = v
else:
raise RuntimeError(f'unexpected registry value for {k}')

for v in ret.values():
v.sort()
Expand Down Expand Up @@ -551,7 +552,7 @@ def process_executor_version(executor_version: str):
for executor_version in all_executor_versions:
process_executor_version(executor_version)

# Verify installed runners (present, correct hashes, latest ⊆ all) and, when
# Verify installed runners (well-formed registry, present, correct hashes) and, when
# requested, precompile them. This is delegated to the manager's `check-install`
# subcommand, which fans out to each active executor's own `check` command
# instead of the executor being precompiled inline above.
Expand Down
53 changes: 51 additions & 2 deletions libs/deps/dependency-urls.json
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,53 @@
"manager-arm64-macos",
"executor-arm64-macos"
],
"fetcher": "fetchzip"
"fetcher": "fetchzip",
"cpes": [
"cpe:2.3:a:gnu:libiconv:1.18:*:*:*:*:*:*:*"
]
},
{
"name": "lua-5.3.6",
"nix_der_name": "source",
"original_url": "https://github.com/lua/lua/archive/75ea9ccbea7c4886f30da147fb67b693b2624c26.tar.gz",
"alternative_urls": [],
"hash": "sha256-5pIGFXxDh7/kYZjz4PNP6icAULit6ufqig+IvaLxYCU=",
"must_preload": false,
"needed-for": [
"x86_64-linux",
"aarch64-linux",
"aarch64-macos",
"manager-amd64-linux",
"executor-amd64-linux",
"manager-arm64-linux",
"executor-arm64-linux",
"manager-arm64-macos",
"executor-arm64-macos"
],
"fetcher": "fetchzip",
"cpes": [
"cpe:2.3:a:lua:lua:5.3.6:*:*:*:*:*:*:*"
]
},
{
"name": "musl-1.2.5",
"nix_der_name": "source",
"original_url": "https://github.com/kraj/musl/archive/0784374d561435f7c787a555aeab8ede699ed298.tar.gz",
"alternative_urls": [],
"hash": "sha256-xNKVsRfEK7TkSiYrxPho9ap8MWGix7nVHLecGW0D7kE=",
"must_preload": false,
"needed-for": [
"x86_64-linux",
"aarch64-linux",
"manager-amd64-linux",
"executor-amd64-linux",
"manager-arm64-linux",
"executor-arm64-linux"
],
"fetcher": "fetchzip",
"cpes": [
"cpe:2.3:a:musl-libc:musl:1.2.5:*:*:*:*:*:*:*"
]
},
{
"name": "rust-channel-stable-2025-03-18",
Expand Down Expand Up @@ -289,6 +335,9 @@
"fetcher": "fetchzip",
"fetcher_args": {
"extension": ".zip"
}
},
"cpes": [
"cpe:2.3:a:sqlite:sqlite:3.24.0:*:*:*:*:*:*:*"
]
}
]
6 changes: 1 addition & 5 deletions libs/libc.nix
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,7 @@
...
}:
let
musl-src = builtins.fetchGit {
url = "https://github.com/kraj/musl.git";
rev = "0784374d561435f7c787a555aeab8ede699ed298";
shallow = true;
};
musl-src = deps."musl-1.2.5";
aarch64-compiler-rt = deps."compiler-rt-aarch64-21.1.5-1";
in
pkgs.stdenvNoCC.mkDerivation {
Expand Down
2 changes: 1 addition & 1 deletion runners/all.nix
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# The accumulated list of every runner across every active executor line, each
# tagged with the `version` of the line it came from:
# [ { id; hash; uid; derivation; version; } ]
# [ { id; hash; uid; derivation; available-after; supersedes; version; } ]
{
host-system ? "x86_64-linux",
deps ? null,
Expand Down
Loading
Loading