Repository navigation
fix(genvm): restore legacy v0.2 contract compatibility - #1762
Conversation
Select the retained v0.2 executor from the first-line source version for ordinary deployments, schema extraction and fee-estimate simulations. Persist the selection and honor existing explicit pins. Use the matching legacy code-storage slot when extracting schemas/code, bootstrapping finalized state and upgrading code; the modern slot aliases the legacy lock list. Validated with 1617 backend unit tests, pinned Ruff/Black checks, and the official GenVM runtime: unchanged legacy deployment, persisted snapshot, schema, read, write and read. Current v0.3 schema remains valid. Full RPC/consensus integration is left to E2E.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
/run-e2e v0.6 all |
|
E2E status was updated. Follow the current E2E and merge checks on this PR. Detailed diagnostics are available internally. |
|
/merge |
|
The request is recorded and waiting for processing. Follow the E2E and merge checks for progress. Do not post a duplicate command. |
|
The merge request cannot proceed. Check reviews, required CI and the selected landing cut. Related changes may require action by their owners; internal details are available to the team. |
2 similar comments
|
The merge request cannot proceed. Check reviews, required CI and the selected landing cut. Related changes may require action by their owners; internal details are available to the team. |
|
The merge request cannot proceed. Check reviews, required CI and the selected landing cut. Related changes may require action by their owners; internal details are available to the team. |
|
Automatic landing has stopped. Earlier merges are not undone. Inspect the current E2E checks, resolve the blocker, then request /merge again. |
|
/run-e2e v0.6 all Resuming the authorized compatibility release to Studio Next/Dev only. Frozen landing cut: #1762 alone, head |
|
E2E status was updated. Follow the current E2E and merge checks on this PR. Detailed diagnostics are available internally. |
|
The new E2E plan resolved the current release branch more freshly than the initial PR metadata: #1750 has already landed, so the tested base is Run 35199189091 tests candidate |
|
/merge Authorized landing of the frozen single-PR cut after full E2E success: https://github.com/genlayerlabs/genlayer-e2e/actions/runs/35199189091 (v0.6, scope=all, stack=all; 174 passed, 0 failed; 107/107 features covered). Head: After landing, release the next unused v0.123 preview version to Studio Next/Dev only. No stable production or staging promotion. |
|
The request is recorded and waiting for processing. Follow the E2E and merge checks for progress. Do not post a duplicate command. |
…acts Studio's policy is that new deployments run on the newest GenVM line, while contracts already on the retained v0.2 line keep working, including the contracts they deploy (factory children, shards). The previous commit routed any new deployment declaring a v0.2-only runner to v0.2, which also admitted new top-level legacy deployments. Restore #1762's header-only rule for top-level deployments, simulated deployments and schema reads, and move runner detection to a separate legacy_executor_selector_for_child_code. At child insertion, consensus pins a deploy child to re:^v0\.2\. through its sim_config when the emitting contract's stored selector is a v0.2 pin (migration backfill, explicit pin or an earlier derived pin) and the child's code is v0.2-only. The child's snapshot factory and accepted effect already honor a sim_config pin, so the selector is persisted on the new contract. An inherited explicit pin still cascades unchanged, and non-deploy children are untouched. Validation: backend unit suite 1773 passed, 7 skipped on a clean checkout; black clean.
…1802) * fix(genvm): route legacy-runner contracts to the retained v0.2 executor The manager picks an executor line by major version only, so every major-0 contract that is not explicitly `# v0.2.x` resolves to the newest line (v0.3). That line ships only its own runners and rejects older ones as `invalid_contract runner malformed`. Contracts with a `# v0.1.x` header, no version line, a Seq header, or a zip runner.json that depend on v0.2 runners could therefore not be deployed on v0.123, and factories could not deploy such children (child deploys carry no executor pin). Also select the retained line when the code's runner spec (parsed like GenVM: zip runner.json, else the leading comment block after an optional version line) declares a runner that appears in an installed `$GENVMROOT/executor/v0.2.*/data/all.json` and in no newer line. Contracts that existed before the multi-line upgrade are already pinned by migration c1d2e3f4a5b6, so no data migration is added. Validation: on studio-next (v0.123.0-rc.8), simulated gen_call deploys of Rally's XID (`# v0.1.0`), CampaignIC (unversioned Seq) and CampaignFactory (zip, py-genlayer-multi) fail with "runner malformed" without a selector and load the runner (constructor TypeError for missing args) with `re:^v0\.2\.`. Registry path checked in the rc.8 jsonrpc image. * test(genvm): keep fake runner ids out of the image pin assertion The backend image greps py-genlayer:<hash> from backend/examples/tests and requires each pin to be installed, so the literal latest/unknownhash ids failed the CI image build. * fix(genvm): limit runner-based v0.2 routing to children of v0.2 contracts Studio's policy is that new deployments run on the newest GenVM line, while contracts already on the retained v0.2 line keep working, including the contracts they deploy (factory children, shards). The previous commit routed any new deployment declaring a v0.2-only runner to v0.2, which also admitted new top-level legacy deployments. Restore #1762's header-only rule for top-level deployments, simulated deployments and schema reads, and move runner detection to a separate legacy_executor_selector_for_child_code. At child insertion, consensus pins a deploy child to re:^v0\.2\. through its sim_config when the emitting contract's stored selector is a v0.2 pin (migration backfill, explicit pin or an earlier derived pin) and the child's code is v0.2-only. The child's snapshot factory and accepted effect already honor a sim_config pin, so the selector is persisted on the new contract. An inherited explicit pin still cascades unchanged, and non-deploy children are untouched. Validation: backend unit suite 1773 passed, 7 skipped on a clean checkout; black clean. * fix(genvm): keep inferred v0.2 pins local and runner sniffing non-fatal Address Codex review on #1802. An inferred child pin was stored in the child's sim_config like an explicit override, so it cascaded to every descendant: a v0.2-derived child deploying current-line code forced that grandchild onto v0.2. Mark inferred pins with genvm_executor_selector_inferred, strip them before cascading, and recompute per deployment. Explicit pins keep cascading as before; to_dict only emits the flag when set. Runner sniffing now runs during consensus message emission, so it must never raise. Treat any unreadable archive (encrypted, unsupported compression) and over-deep JSON as "no declared runners", and walk the parsed spec iteratively. Invalid code still reaches GenVM's invalid-contract path. Validation: new tests fail on the previous head (6) and pass here; backend unit suite 1777 passed, 7 skipped on a clean checkout.
* feat: add v0.6 fee accounting
* fix(ci): update studio test expectations
* fix(ci): sync frontend lock optional peers
* fix(ci): address studio sonar quality gate
* fix(ci): clear studio sonar annotations
* test(studio): assert genvm fee context
* test(studio): cover message value fee guards
* ci(studio): scope sonar coverage to backend reports
* test(studio): cover unbacked external message value
* fix(studio): discard messages after budget exhaustion
* feat(studio): show genvm metered message fees
* fix: build studio with matrix genvm tag
* fix: require explicit genvm tag for studio builds
* fix: enforce studio fee execution floor
* ci: add studio branch policy guard
* [codex] Fix finalization deadlock and surface stuck-head context (#1667)
* fix: surface stuck-head alert context
* fix: avoid finalization claim deadlock
* ci: narrow release workflow secrets
* ci: keep codecov upload non-blocking
* test: cover recovery storm health summary
* test: cover no-progress health scan
* test: cover no-progress health fallbacks
* test: cover detailed consensus health endpoint
* fix(fees): user-facing money correctness + canonical RPC alignment (#1669)
* fix(fees): user-facing money correctness + canonical RPC alignment
Money correctness (designer rulings — all user-visible):
- appeal bonds now settle: bond × 3/2 credited to the appellant on successful
appeal, explicit forfeiture on failure, 1:1 return on cancel; payouts credited
to the recorded appellant (not the tx sender), idempotent across re-settles.
Previously every bond was silently destroyed.
- zero-value submitAppeal no longer bypasses min-bond validation (gasless mode
unaffected)
- unused pre-funded rotations refunded: rounds bill min(funded, actual
LEADER_ROTATION events) + 1 leader slots instead of always rotations[i]+1
- LEADER_TIMEOUT rounds charge the leader allocation at 50% (extensible
per-outcome multiplier table)
- round indexing no longer counts LEADER_ROTATION events as consensus rounds
(shared helper backend/consensus/history.py) — fixes settlement ladders and
appeal-bond quotes when appealRounds > 0
Canonical RPC alignment (cross-backend receipt schema, superset rule):
- gen_getTransactionStatus returns {status, statusCode} with canonical v0.6
ITransactions ordinals (0-14, incl. VALIDATORS_TIMEOUT=12, LEADER_TIMEOUT=13)
- transaction payloads expose top-level txExecutionResult(Name) and a canonical
fees object (deposit/userValue/distribution/locked/consumed as decimal
strings; null when gasless)
CI: GenVM precompile cache key now uses the actual GENVM_TAG (env or .env),
failing loud when absent — the Dockerfile grep had been yielding an empty key
since GENVM_TAG lost its default.
* fix: keep transaction value an int; soften GenVM cache-key guard; black
- top-level 'value' returns to plain int per the numeric-columns contract
(tests/db-sqlalchemy/test_numeric_types.py) — the blanket
_json_safe_numbers stringification from the fee-accounting merge broke it
for values > 2^53; the breakage predates this PR but was masked by CI path
filters until now. Big-int-safe consumers read the canonical decimal-string
fees object instead.
- GENVM_TAG cache-key step: warn + 'untagged' sentinel instead of hard-fail —
the load-test and integration jobs run prebuilt images (--no-build) and
legitimately have no tag; hard-failing broke them while the warning keeps
the degradation visible.
- black: 2 files reformatted (pre-commit gate).
* style: restore calldata.py to pinned black 24.4.2 formatting
Local black 25.1.0 had reformatted it differently from CI's pre-commit pin;
the file was never part of this PR's functional changes.
* ci: keep main forwarded to active dev branch (#1671)
* fix(ci): default GENVM_TAG for standalone PR builds (#1673)
* fix(ci): default GENVM_TAG for standalone PR builds
docker-compose hard-requires GENVM_TAG since 963f9359, but no PR
workflow sets it and .env.example ships it empty, so any run that
reaches the buildx bake step fails with 'GENVM_TAG is required'
(first surfaced on #1672). Default to the GENVM_TAG repo variable,
falling back to the current GenVM release, exported both to .env and
the job env so bake and compose interpolation agree.
* fix(ci): pin default GENVM_TAG to v0.3.0-rc3 (v0.6 fee host protocol)
The Load Tests state-integrity step timed out with the Counter deploy
stuck in PROPOSING (run 27292725606). Stack logs show every GenVM run
failing instantly:
genvm manager /genvm/run failed: 500 {'error': 'unknown variant
`execution_budget_per_round`, expected one of: Internal, External'}
The studio v0.6 fee code sends message_fee_allocation nodes in the
GenVM v0.3.0-rc3 schema (MessageFeeAllocationNode: message_type/
parent_index + flat MessageFeeParams). GenVM v0.3.0-rc4 replaced that
with the tagged Internal/External MessageAllocationNodeParams enum, so
the rc4 manager rejects the payload, the leader burns its full 600s
slot, and deploys end in LEADER_TIMEOUT (or sit in PROPOSING until the
test gives up).
Default the CI GenVM tag to v0.3.0-rc3 until the host protocol is
migrated to the rc4 schema. vars.GENVM_TAG still takes precedence.
* feat(ci): bind GenVM to a branch — source-build path + main host-protocol adaptation (#1674)
* fix(ci): default GENVM_TAG for standalone PR builds
docker-compose hard-requires GENVM_TAG since 963f9359, but no PR
workflow sets it and .env.example ships it empty, so any run that
reaches the buildx bake step fails with 'GENVM_TAG is required'
(first surfaced on #1672). Default to the GENVM_TAG repo variable,
falling back to the current GenVM release, exported both to .env and
the job env so bake and compose interpolation agree.
* fix(ci): pin default GENVM_TAG to v0.3.0-rc3 (v0.6 fee host protocol)
The Load Tests state-integrity step timed out with the Counter deploy
stuck in PROPOSING (run 27292725606). Stack logs show every GenVM run
failing instantly:
genvm manager /genvm/run failed: 500 {'error': 'unknown variant
`execution_budget_per_round`, expected one of: Internal, External'}
The studio v0.6 fee code sends message_fee_allocation nodes in the
GenVM v0.3.0-rc3 schema (MessageFeeAllocationNode: message_type/
parent_index + flat MessageFeeParams). GenVM v0.3.0-rc4 replaced that
with the tagged Internal/External MessageAllocationNodeParams enum, so
the rc4 manager rejects the payload, the leader burns its full 600s
slot, and deploys end in LEADER_TIMEOUT (or sit in PROPOSING until the
test gives up).
Default the CI GenVM tag to v0.3.0-rc3 until the host protocol is
migrated to the rc4 schema. vars.GENVM_TAG still takes precedence.
* feat(docker): add GENVM_REF source-build path for GenVM images
GENVM_TAG (release tarball) and GENVM_REF (git ref/SHA, built from source)
are now mutually exclusive build args; exactly one must be set. The source
path adds a nixos/nix builder stage that replicates genvm's own release
recipe (.github/workflows/release.yaml + support/ci/pipelines/build.sh):
patch flake-config.json with the resolved SHA, prefetch pinned deps,
nix build manager-<arch>-linux / executor-<arch>-linux / runners-all, and
merge the outputs into the same /genvm layout the tarballs produce.
Prebuilt runner tars are preloaded from genvm's public GCS cache
(fixed-output derivations) so runners-all rarely compiles anything.
The tarball path is unchanged; post-install runs with
--default-download false only for source builds (no release to download
the executor from).
* feat(ci): support vars.GENVM_REF — resolve to SHA, scope buildx cache by it
When the GENVM_REF repo variable is set (e.g. 'main'), the workflows
resolve it to a commit SHA via the GitHub API, pass GENVM_REF=<sha> as
the build arg instead of GENVM_TAG, key the genvm-precompile cache on
ref-<sha>, and move the buildx gha cache into a scope=genvm-<sha>
namespace so image layers invalidate exactly when genvm moves.
Behavior with the variable unset is unchanged (GENVM_TAG path).
* feat(genvm): adapt host protocol to GenVM main (CON-549 fee schema)
Studio spoke the rc3-era GenVM host protocol; genvm main (rc4+, #310)
restructured the message-fee surface and the manager rejects our
/genvm/run payload with 500 "unknown variant 'execution_budget_per_round',
expected one of: Internal, External".
Request (message_fee_allocation): flat parent-pointer nodes with
message_type InternalAccepted/InternalFinalized/External and a 4-field
fee_params dict become a nested tree ('children') with 'on'
accepted|finalized and an externally tagged params enum —
{'Internal': {7 fields incl. max_price_gen_per_time_unit,
storage_fee_max_gas_price, receipt_fee_max_gas_price}} or
{'External': {gas_limit, max_gas_price}}. The three price caps are
sourced from Studio's fees_distribution (they are not part of the
canonical per-node 5-field feeParams ABI). Non-root allocations are now
sent as children of their roots so GenVM can emit correct subtrees
(genvm executor/crates/common/src/domain/fees.rs,
modules/implementation/src/manager/run.rs, tests/runner/origin/fees.py).
Response (emissions): PostMessage/DeployContract/EthSend now carry
fee_params as a struct and an ABI-encoded allocation 'subtree'
(8-field internal params ABI). These are translated back at the GenVM
boundary to Studio's canonical surfaces: 5-field internal / 2-field
external feeParams bytes and the allocationSubtree node list. Studio's
canonical fee ABI and fees.consumed reporting are unchanged.
backend/node/genvm/origin/* is re-synced with genvm main's
tests/runner/origin (new fees.py vendored module).
* fix(fees): restore Mode-1 child bucket seeding; skip phase check for external allocations
Two regressions from b88e0594 ('enforce studio fee execution floor',
June 4) that broke TestMessageFeeModes (added with the original v0.6 fee
accounting, #1649):
- create_child_fee_accounting zeroed the child's message-fee bucket when
the message carried no allocation subtree, refunding the declared-budget
remainder instead. Mode-1 children have no subtree by definition but
must still receive the remainder as their own message bucket
(FeeBudgetTransferred custody semantics). The leaf-refund expectation
test from b88e0594 is updated to the seeding semantics.
- fill_message_fee_payload_from_allocation phase-checked (onAcceptance)
external allocations; external messages have no accepted/finalized
lifecycle (GenVM's matches_external ignores 'on' for the same reason).
* fix(docker): disable nix syscall filtering in the genvm builder stage
nix's seccomp BPF filter fails to load under qemu/Rosetta emulation
('unable to load seccomp BPF program: Invalid argument'), which blocks
cross-platform local source builds (e.g. linux/amd64 on Apple Silicon).
Harmless on native hosts — the nixos/nix image already runs with the
build sandbox disabled.
* feat(docker): GENVM_BUILD_CORES knob for the genvm source build
Caps nix 'cores' for the genvm builder stage (default 0 = all). A
full-parallel rust build can OOM memory-constrained or emulated
builders (observed: cargo -j10 under Rosetta in a 16GB VM kills the
nix daemon).
* feat(ci): bind GenVM via checked-in third_party/genvm/version, env override
Replaces the repo-variable mechanism: the GenVM binding is now a
versioned file (same convention as genlayer-node), flipped to the
release tag at release cut. Resolution order in CI:
1. GENVM_TAG / GENVM_REF already in the environment (e2e matrix /
release train) wins; both set is a hard error.
2. Otherwise third_party/genvm/version: ^v[0-9] = release tag
(tarball path), anything else = git ref (source build, resolved
to a SHA for exact cache invalidation). Missing/empty file is a
hard error rather than a guess.
The file says 'main' on v0.123-dev, so PR CI builds genvm from source
and validates the binding with no out-of-band setup.
* fix(docker): derive genvm executor version from the bound ref's manifest
genvm's patch-manifest requires the executor version to extend a version
the committed manifest knows (base-match or previous-patch lookup);
v0.0.0-dev<sha> matched nothing and failed the manager/executor builds
('Could not find any previous version for tag'). Derive the highest
vX.Y.Z from modules/install/data/manifest.yaml of the ref being built
and tag the source build vX.Y.Z-dev<sha> (verified against genvm's
support/scripts/patch-and-check-manifest.py).
* fix(docker): preserve exec bits when merging genvm source-build outputs
cp --no-preserve=mode stripped the nix-store exec bits, so the base
stage failed with '/genvm/bin/post-install.py: Permission denied'
(exit 126) after an otherwise successful source build on CI. Preserve
modes, then chmod -R u+w so post-install can write venvs/cache dirs.
Also correct the builder comment: arm64 builds natively (the flake
target follows TARGETARCH; runners are universal wasm from the GCS
cache either way).
* fix(docker): native arm64 genvm source builds + binding parse hardening
- Work around genvm upstream bug: support/rust.nix patchelf's its rust
toolchain with a hardcoded x86-64 ELF loader, breaking native
aarch64-linux builds ('.cargo-wrapped: cannot execute'). On arm64 the
builder rewrites it to ld-linux-aarch64.so.1, guarded to skip cleanly
once fixed upstream and to fail loudly if the line multiplies.
- nixos/nix ships no sed: install gnused alongside python3.
- Harden the workflow binding parse: CRLF/whitespace trim + optional
quoted values in third_party/genvm/version.
Validated locally: full backend image builds natively on Apple Silicon
(genvm stage 7m20s, GENVM_BUILD_CORES=6, no OOM); executor reports
genvm v0.3.0-dev<sha>-aarch64-linux-release. (Implementation reviewed
via codex.)
* fix(docker): skip lief bin-patch for source-built genvm
post-install's lief RPATH rewrite SIGABRTs on the zig/musl-linked amd64
executor built from source ('Can't read got entries' / 'Node not found.
Can't resize the section'). Source-built outputs already carry
$ORIGIN-relative rpaths from genvm's own nix patch-rpath, so the lief
pass is redundant on this path; bin-check still verifies the binaries
execute. Tarball path unchanged.
* fix(genvm): retry transient 'modules not running' 500 from the manager
Studio stops+restarts the genvm llm/web modules on every validator
change; a /genvm/run landing in that window gets 500 'modules are
required but not running' from the manager. The check exists in rc3
identically, but genvm main's module restart window is wide enough that
mock-validator integration tests (which reconfigure validators right
before each deploy) hit it near-deterministically — txs park in
PROPOSING until timeout (run 27311560490).
Route only this transient 500 (plus the sibling 'not all are running')
into the existing GENVM_MANAGER_RUN_RETRIES retry loop via a dedicated
GenVMManagerRetryableError; every other non-200 keeps failing fast
(e.g. protocol errors must not be retried). Retry stats record the
reason. (Implemented via codex; +2 parametrized regression tests.)
* fix(genvm): match GenVM main's 32-byte U256 fuel framing in host protocol
GenVM main widened fuel values to U256 in the host wire protocol;
base_host.py still used rc3's 8-byte framing in two places:
- REMAINING_FUEL_AS_GEN reply: we sent 8 bytes, the executor reads 32
(executor/src/host/mod.rs remaining_fuel_as_gen at v0.3.0-devef04f7f).
The executor blocked on the missing 24 bytes while our host loop
waited for the next method - a silent read-read deadlock. It sits in
the exec_prompt path (wasi/genlayer_sdk.rs fetches remaining fuel
right before the LLM module call), so every nondet execution parked
in PROPOSING until LEADER_TIMEOUT while deterministic contracts were
unaffected (runs 27311560490, 27314907925).
- CONSUME_FUEL request: executor writes a 32-byte U256, we read 8,
desyncing the stream whenever it is invoked.
Locally wizard_of_coin went from a 10-minute hang to passing in ~11s.
* fix(consensus): don't kill workers on transient NO_PROVIDER_FOR_PROMPT leader errors
The llm module reports NO_PROVIDER_FOR_PROMPT as fatal, but in Studio it
is transient/environmental: every validator change stops+restarts the
llm module, so a leader run claimed inside that window (or on a stack
without a real API key) sees a provider table lacking its node address.
The fatal-leader circuit breaker stopped the worker in that case; with
mock-validator tests reconfiguring validators before every test, all
workers died one by one and the queue stalled silently (health:
worker_task_died). The tx already has a bounded retry budget (recovery
cycles escalate to CANCELED), so keep the worker alive and back off
briefly (GENVM_TRANSIENT_FATAL_BACKOFF_S, default 3s) before resetting
so retries don't burn through their cycles inside the same restart
window.
* test: modernize integration contracts to GenVM v0.3 SDK
The stack now source-builds GenVM from main, where the floating runner
aliases py-genlayer:test / :latest / py-genlayer-multi:test resolve to
the genvm-main runner whose SDK no longer star-exports gl - every
fixture still on an alias died at import with NameError: name 'gl' is
not defined (and the upgrade suite cascaded into Contract-not-found).
- Pin all alias fixtures to the v0.3.0-rc3 runner already used by every
green contract (py-genlayer:1jb45aa8..., py-genlayer-multi:06zyvrl...):
8 icontracts fixtures, 5 inline upgrade-test fixtures, and
multi_file_contract/runner.json. Bodies were already valid v0.3 SDK.
- log_indexer: repin py-lib-genlayer-embeddings from the vanished
09h0i209... hash (invalid_contract) to 0bmbm3cy..., whose VecDB takes
an explicit Distance type parameter - annotate with
EuclideanDistanceSquared.
- Allow overriding the integration RPC endpoint via TEST_JSONRPC_URL
(request helper, conftest rate-limit probe, upgrade tests, utf8
roundtrip test). The previous hardcoded localhost:4000 made the suite
untestable against a non-default stack; CI behavior is unchanged as
the variable is unset there.
Contract repins implemented via codex; verified against a local stack
built from this branch (mock validators): 29/31 previously-failing
tests pass; genvm_smoke needs a real LLM key (CI has one) and
log_indexer is slow-but-valid locally (schema verified via RPC).
* style: black-format worker.py backoff hunk
* fix(genvm): discard storage changes reported by readonly executions
GenVM main reports result_storage_changes for readonly (view) executions
too - e.g. genlayer-embeddings VecDB lazily initializes its cover-tree
storage on first access, which can happen inside a view knn call.
provide_result applied those changes into the readonly state proxy and
tripped 'assert not self.readonly' in storage_write, surfacing as
'-32603 Internal error' on every gen_call to such a contract
(test_log_indexer). View-execution storage changes are ephemeral VM-side
effects: skip applying them.
* test: upgrade integration contracts to genvm main SDK
Supersedes the rc3 repin (098bbb8a): instead of pinning fixtures
backwards, every integration contract now uses GenVM main's SDK idiom,
pinned to main's concrete runner hashes (no test/latest aliases):
- py-genlayer:1zr6nqk597d97kg0dyxg0shhrykx5v02zjgnyrajapy4wlqvfvwh
- py-genlayer-multi:0jw6688wgnbqghr2wwg2mqkf2ryb5if7w6bp5zcwbq6zlks5ybg4
- py-lib-genlayer-embeddings:0wcvi35grdr47ynkckzriz5sjn5080w2njk7v2cqx3xpp6p1989y
Idiom changes (mirroring genvm's own tests, e.g.
tests/cases/semi-stable/llm/woc.py):
- import genlayer as gl + from genlayer import * (gl is no longer
star-exported); class X(gl.contract.Contract)
- gl.contract.get_at / gl.contract.deploy (were gl.get_contract_at /
gl.deploy_contract); raise gl.vm.UserError (gl.Rollback is gone)
- eq_principle.prompt_comparative principle arg is positional-only
- @allow from genlayer.storage (was @allow_storage)
- exec_prompt(..., response_format='text')
- embeddings VecDB takes an explicit Distance type param
Backward-compat canary: examples/contracts/user_storage.py deliberately
stays on the rc3 runner (py-genlayer:1jb45aa8..., old idiom) so
test_user_storage keeps proving old-runner contracts execute on the
genvm-main executor. (The executor parses the whole leading comment
block as version+runner JSON, so the canary note sits after a blank
line.)
gltest 0.24.0 only discovers contracts with the legacy gl.Contract base;
tests/integration/gltest_compat.py (activated in conftest) extends the
AST matcher to accept both bases until the genlayer-test pin moves past
0.29.2 (upstream main already matches gl.contract.Contract).
log_indexer specifics: the main embeddings runner pairs with the main
py-genlayer (the old 09h0... hash no longer exists); similarity values
recalibrated for its EuclideanDistanceSquared metric; update_log now
locates entries by plain iteration because cover-tree knn from a write
method currently trips GenVM main's deterministic-mode float trap
(wasm_trap DeterministicMode) - views still exercise knn.
Bulk rewrite implemented via codex; verified against a local stack built
from this branch: previously-failing set + upgrade suite + canary all
green locally except genvm_smoke (needs a real LLM key; CI has one).
* style: appease autoflake on star-imports; skip genvm-lint for main-runner contracts
- autoflake (pre-commit) removes 'from genlayer import *' where no
star-exported name is used - accepted for the 7 contracts that only
use the gl namespace.
- Released genvm-linter (<=0.11.0) cannot load the genvm-main SDK layout
('No module named genlayer.py'); the lint workflow now skips contracts
pinned to the main runner hash until a compatible linter ships
(genvm-linter dxp-694), and keeps linting rc3-era contracts (canary,
load/direct fixtures).
* test(unit): cover host fuel-width framing, transient-fatal backoff, readonly storage discard
Targeted unit tests for the Sonar new-code coverage gate (PR #1674):
- test_host_loop_fuel_widths: drives host_loop over a socketpair and pins
the GenVM main wire framing - CONSUME_FUEL reads a 32-byte LE U256
(value > 2**64) and REMAINING_FUEL_AS_GEN replies OK + exactly 32 LE
bytes. Regression tests for the read-read deadlock fixed in dfdee12c.
- test_worker_transient_fatal: NO_PROVIDER_FOR_PROMPT fatal leader errors
reset the tx, back off (3s default), and keep the worker alive; other
fatal causes still stop the worker; validator-side errors are left to
consensus.
- test_provide_result_readonly: provide_result discards result storage
changes for readonly (view) executions and applies them for writable
ones (49db980d).
Also exclude examples/contracts/** from Sonar coverage: they are GenVM
intelligent-contract fixtures executed inside the GenVM, never under the
backend test runner (18 of the 38 'uncovered new lines' were contract
fixture lines).
Full tests/unit suite in-container: 903 passed, 1 skipped.
* feat(fees): report time-unit consumption on receipts (unified consumed shape) (#1672)
The canonical fees.consumed object now reports how many protocol time
units the execution actually used, per round and in total:
- leaderTimeunitsUsed / validatorTimeunitsUsed: decimal-string totals
across all rounds (validator total includes the leader's
validator-function execution)
- perRound: [{round, consensusRound, leaderTimeunits, validatorTimeunits,
maxValidatorTimeunits}] - maxValidatorTimeunits is the single largest
validator execution that round, the number to compare against the
per-validator validatorTimeunitsAllocation
Unit mapping (documented at backend/consensus/history.py): 1 time unit ==
1 second of GenVM wall-clock runtime (consensus spec glossary; Studio's
DEFAULT_INITIAL_TIME_UNITS_ALLOCATION = 10*60). Consumption is derived
from each receipt's measured processing_time (ms), rounded UP per
execution - never raw seconds.
Computation is a pure helper (time_unit_consumption) over the persisted
consensus_history round entries: leader-rotation events fold into the
next completed round, with a fallback to consensus_data when no history
rounds exist. Backward compatible: new keys only; gasless transactions
still return fees: null.
This is the reference shape for the node to mirror 1:1; consumer:
gltest --fee-profile (genlayer-testing-suite #94) fee-profile
suggestions for leader/validator time-unit allocations.
* docs: add branching guide (#1675)
* fix(rpc): gen_getTransactionStatus returns the legacy string again (#1677)
ad77f6b7 changed the response to {status, statusCode} — a breaking
change for deployed apps that parse the string (Rally polls it on every
transaction). The string shape is a compatibility contract; the dict
moves to a new additive method, gen_getTransactionStatusDetails. No
in-tree consumer used the dict.
* test(ci): gasless-mode integration coverage (#1676)
* test(ci): gasless-mode integration coverage
Rally's hosted studio runs with all three price envs at zero
(GENLAYER_STUDIO_GEN_PER_TIME_UNIT / STORAGE_UNIT_PRICE /
RECEIPT_GAS_PRICE). Only unit tests covered that configuration; no
integration/CI job ever ran a zero-price stack, so fee-path changes
could silently break the hosted deployment.
- tests/integration/test_gasless_mode.py (@pytest.mark.gasless):
pins sim_getFeeConfig enabled=false + zero policy prices, fee-less
deploy/write/read via the SDK, canonical receipt fees: null, the
sim_estimateTransactionFees surface (the RPC genlayer-js
estimateTransactionFees hits), and the tolerant behavior that
explicit fee params on a gasless stack are accepted (required
deposit is 0), not rejected.
- pytest.ini: register the gasless marker and exclude it from the
default sweep (same convention as error_handling).
- backend_integration_tests_pr.yml: new test-gasless job reusing the
bake/compose pipeline with the three price envs forced to 0, a
readiness assert on sim_getFeeConfig, and a serial -m gasless run.
Buildx cache is read-only in this job so it rides the main test
job's cache without racing its writes.
Verified locally against a zero-price compose stack (arm64,
GENVM_REF source build): 2 passed.
* test(ci): appease Sonar hotspots in the gasless job
- SHA-pin docker/setup-buildx-action and docker/bake-action (S7637)
- drop the explicit http:// scheme from the local readiness curl (S5332);
curl defaults to http and the endpoint is runner-local only
* refactor(fees): reduce canonical fee helper complexity
* test(ci): stabilize gasless and oracle CI
* fix: keep worker liveness above tx timeout
Prevent Kubernetes liveness from restarting workers before the configured transaction timeout can own stuck transaction recovery.
Validated by CI, load test, and focused worker health unit tests.
* fix: surface stuck finalization health context
Admin merge: green CI, targets v0.123-dev, part of the fee-train Studio cleanup.
* ci: consume shared GenVM runtime in Studio E2E builds (#1693)
* ci: accept shared GenVM runtime in studio builds
* test: update Studio GenVM v0.3 contract fixtures
* ci: scope Studio workflow write permissions
* ci: exclude legacy e2e workflow from Studio sonar
* fix: patch prebuilt GenVM in Studio images
* fix: update log indexer vectors by id
* fix: track log indexer vector ids
* fix: avoid deterministic VecDB deletes
* fix: update log indexer entries in place
* feat: port v0.121 snapshot pruning, LLM metrics/router, and robustness fixes to v0.123-dev (#1699)
* fix: release worker claims before shutdown deadline (#1682)
* feat: add llm router provider capability
Adds inert llm-router/policy:auto provider capability and focused provider config forwarding tests.
Includes the sim_updateValidator config-forwarding fix needed for provider config propagation.
* fix: tolerate dict leader receipts
* feat: report configured llm token metrics (#1686)
* [codex] Archive terminal snapshots before pruning (#1680)
* feat: archive terminal snapshots before pruning
* test: cover terminal snapshot pruning
* feat: archive pruned contract snapshots
* test: verify snapshot archive before pruning
* test: exclude archive migrations from coverage
* ci: exclude archive migrations from sonar coverage
* test: cover snapshot archive retrieval paths
* feat: add parallel snapshot pruning drain
* ci: allow preview builds without latest push
* docs: add snapshot pruning rollout checklist
* fix: read raw gcs snapshot archives (#1687)
* fix: reuse existing gcs snapshot archives (#1688)
* perf: split snapshot pruning pipeline
Split terminal snapshot pruning into archive, verify, and prune phases so large drains can upload first, verify separately, then delete only verified hot snapshots.
* fix: bound terminal snapshot archive scan
Bound the archive candidate scan before filtering already-archived rows so split archive mode does not devolve into a broad anti-join on production-sized data.
* [codex] Defer archive snapshot payload reads (#1692)
* fix: defer archive snapshot payload read
* test: update pruning fake for deferred payload query
* chore: drop dead transaction_hash_variant assignment
The original v0.121 commit (75f86bea) removed this local in
_gen_call_with_validator; the removal hunk was lost while resolving
conflicts against dev's fee-simulation changes during the port.
* fix: bound usage metrics percentages
Port the v0.121 usage metrics percentage-bounding hotfix to v0.123-dev.
Bounds system health memoryUsage and cpuUsage before sending metrics so out-of-range process CPU samples cannot make the usage metrics API reject health reports.
Source:
- v0.121 PR: #1700
- v0.121 release tag: v0.121.16
- Released v0.121 commit: 65042fa8699b0f42cb92f3dec7b0fb1a29b0ec23
- Original hotfix commit: 283c68f2171187f3f3b1016ac68c9fcd4d6371fe
Validation:
- .venv/bin/python -m pytest tests/unit/test_usage_metrics_service.py
- GitHub Actions: CI, Unit Tests, Load Test, Branch Policy, CodeQL, pre-commit all success
* fix: coalesce duplicate gen_call reads (#1702) (#1703)
(cherry picked from commit cdba5089e53400f8ef959b0ff85dadcdc2b2055a)
* feat(genvm): port studio to genvm-manager v0.3 (#1697)
* fix(pruner): stop declaring Content-Encoding on snapshot archives (#1708)
The archive bodies are already gzip-compressed and their keys end in
.json.gz, so declaring Content-Encoding: gzip on the stored object adds
nothing for our own readers, which fetch stored bytes via
download_as_bytes(raw_download=True) and call gzip.decompress()
themselves.
It does, however, make GCS serve the objects with decompressive
transcoding, which breaks every external reader:
- a plain GET returns the decompressed byte count with no
Content-Length, while ListObjects advertises the stored size
- Range requests are silently ignored, returning 200 instead of 206
AWS DataSync fails on both counts. A task over the prd studio snapshot
archives listed 233 objects and errored on all 233, transferring 0
bytes. rclone ("corrupted on transfer: sizes differ") and aws s3 cp
against the S3-compatible endpoint fail the same way. This is also what
PR #1687 worked around from the read side by adding raw_download=True
rather than removing the header.
The S3 writer's ContentEncoding is dropped for the same reason. S3 does
not transcode on GET, so nothing server-side changes there, but the
header invites HTTP clients to decompress bodies that readers already
handle themselves.
Objects written before this change still carry the header and need a
metadata backfill to become copyable:
gcloud storage objects update "gs://<bucket>/<prefix>/**" \
--content-encoding="" --project=<project>
* fix(genvm): download GenVM releases instead of building from source (#1706) (#1712)
* fix(genvm): acquire GenVM by release download instead of nix build
Studio always built GenVM from source because the pin file held a
floating "main". That nix build fails nondeterministically on a
fixed-output derivation, so add explicit prebuilt/source/release modes
and default to downloading a pinned genvm-manager release.
* fix(ci): precompile GenVM before starting the stack
The genvm-manager bundle carries two executor lines, so precompiling both
on a cold cache overruns the healthcheck window and the container is
reported unhealthy. Precompile in a one-off container first and cache the
result across runs.
* chore(ci): replace SonarCloud with ruff + Codecov (#1710)
* chore(ci): remove SonarCloud
The quality gate measured the wrong diff. Dev branches were never
branch-analysed, so PRs against them were scored against a stale main:
the same 8-line fix passed on one base and failed on another. It was
never a required check, and 9 of the last 60 merged PRs merged red.
CodeQL and Semgrep already cover the security rules.
* ci(coverage): send backend coverage to Codecov
Backend coverage.xml had exactly one consumer, and removing SonarCloud
left it unread. Codecov scores the real PR diff instead of a stale
branch baseline, which is what the old gate got wrong.
* ci(codeql): rescan pull requests on push
Without synchronize, CodeQL only ran when a PR opened or reopened, so the first commit's result kept showing while later commits went unscanned.
* ci(lint): add ruff and fix the bugs it found
Backend linting was black and autoflake only. Ruff catches three asyncio tasks created without a retained reference, which the event loop can garbage-collect mid-flight so a log publish silently never happens.
* fix: correct defects found while retiring SonarCloud (#1711)
CORS advertised credentialed access to any origin and did not decorate
rate-limit rejections; consensus events were gated on a comparison that
never matched; readiness jitter was identical across replicas.
* fix: preserve serialized receipt errors in usage metrics (#1713)
* test: reproduce serialized receipt metrics bug
* fix: read serialized receipt execution results
* fix: preserve exact faucet amounts in wei (#1714)
* perf(studio): share GenVM service image layers (#1718)
Consolidate Studio backend and consensus-worker images onto a common GenVM runtime and service base so AWS E2E pulls each large runtime layer once.
* chore: fix source builds (#1717)
* chore: add nix cache (#1719)
* feat(consensus): make validator execution concurrency configurable (#1723)
Replace the hardcoded asyncio.Semaphore(8) in CommittingState with the CONSENSUS_VALIDATOR_MAX_CONCURRENT env var (default 8, min 1) so large committees (23-47+ validators) are not serialized into ~6 rounds of 8. Port of the v0.121 change. Refs #1721
* ci: skip lint for contracts pinned to the v0.6 fee-era runner (#1725)
genvm-linter 0.11.0 (latest release) does not bundle the
py-genlayer:9b8kjyda2... runner that every v0.123-dev example contract
now pins, so 'Lint Intelligent Contracts' fails with
"filename 'runners/py-genlayer/9b/8kjy....tar' not found" on any PR
touching contracts. Extend the existing unsupported-runner skip filter
(same mechanism already used for the genvm-main runner) until a linter
release bundles the new SDK (genvm-linter dxp-694).
* fix(consensus): appeal claim omits contract_snapshot/consensus_history, wiping contract state on successful validator appeal (#1724)
* fix(consensus): return contract_snapshot and consensus_history from claim_next_appeal
claim_next_appeal's raw UPDATE ... RETURNING omitted the
transactions.contract_snapshot and transactions.consensus_history
columns, so every appeal transaction was rebuilt via
Transaction.from_dict with contract_snapshot=None and an empty
consensus_history. On a successful validator appeal,
process_validator_appeal then fell into its "no snapshot" branch and
restored accepted_state = {} — for non-deploy write transactions this
wiped the contract's accepted state including the code slot, after
which the re-run failed with invalid_contract absent_runner_comment
and subsequent appeals self-confirmed the error. The missing
consensus_history additionally let appeal committees re-draw
already-used validators, and appeal validators verified against
post-execution state instead of the pre-execution snapshot.
Two changes:
- worker.py: add both columns to the RETURNING list and the returned
appeal_data dict (same raw-JSONB shape as _parse_transaction_data).
- base.py: defense in depth in process_validator_appeal — for
non-deploy transactions with a missing snapshot, re-fetch it via
transactions_processor.get_transaction_by_hash (which hydrates
archived snapshots); if it is still unavailable, log an error and
skip the state restore instead of writing {}. Deploy rollback
behavior is unchanged.
Adds unit tests covering the claim dict shape, the RETURNING clause,
Transaction.from_dict hydration of a claimed appeal, and all four
restore branches of the appeal rollback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(consensus): generate claim queries from a shared column manifest
Review follow-ups for this PR, plus the architectural fix for the bug
class it exposed (hand-maintained RETURNING lists + silent
Transaction.from_dict defaults = omissions indistinguishable from absent
values):
- worker.py: all three claim queries (transaction, finalization, appeal)
now generate their RETURNING clause and row->dict conversion from one
shared column manifest, so query and dict can never drift apart. Heavy
state columns (contract_snapshot has been observed at tens of MB) are an
explicit opt-in group: appeals include them, pending/finalization
deliberately omit them.
- base.py: PendingState now refreshes the transaction from the DB before
computing pre-effects; previously decide_pending_pre received
claim-built appeal flags that silently default to False for
re-processed appeals (same drift class, latent since the params are
currently unused). Renamed previous_contact_state ->
previous_contract_state, and the skipped-restore path now emits a
consensus_event ERROR so operators see it, not just the log.
- tests/unit/consensus/test_claim_column_manifest.py: pins each claim's
SQL to its manifest, pins the appeal claim to the state columns
(incident regression), and statically extracts every key
Transaction.from_dict consumes to enforce that each claim either
returns the column or explicitly allowlists the omission with a
rationale - a new from_dict field now fails tests until a conscious
per-claim decision is made.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Edgars <edgars@genlayerlabs.com>
* feat(genvm): calls between different majors of genvm (#1716)
* feat: support v0.2 v0.3 interop
* fix(genvm): accept a base64 consumed_result from the manager
* feat: port contract storage quota to v0.123-dev (#1729)
Ports the per-contract daily snapshot storage quota and actual snapshot-size accounting correction to v0.123-dev.
* fix: redact validator private keys from RPC (#1731)
* feat(rate-limit): port read bucket, usage headers and path-based API keys to v0.123 (#1743)
* feat(rate-limit): meter cheap reads separately and expose usage headers (#1741)
Rate limiting was method-agnostic: the middleware checked the quota before
the JSON-RPC body was parsed, so `gen_getContractCode` — a single indexed
read — cost exactly what a write fanning out to LLM validators cost. That
forces the limits to be sized for the expensive case, which starves the
cheap read traffic that batch tooling generates.
Reads that never enter the GenVM now meter into their own bucket at
RATE_LIMIT_READ_MULTIPLIER (default 10x) of the tier limits. The standard
bucket keeps its key shape and numbers, so limits in flight at deploy time
carry over rather than resetting.
The allowlist is maintained by hand and deliberately conservative. Two
methods that read like lookups are excluded because they build a Node backed
by a GenVMManager (gen_getContractSchema, gen_getContractSchemaForCode), and
eth_call is excluded for the same reason despite appearing in the
DISABLE_INFO_LOGS_ENDPOINTS env list that otherwise looks like the natural
source for this. Anything ambiguous — unparseable, oversized, or a batch with
one expensive member — is charged to the stricter bucket.
Responses now carry X-RateLimit-Limit/Remaining/Reset/Window/Bucket for the
window closest to exhaustion, so clients can pace themselves instead of
discovering the ceiling by hitting it. Reset is the time until the oldest
entry ages out, which is the honest answer for a sliding window. These are
listed in the CORS expose_headers, without which browsers hide them from JS.
Finally, gen_getContractCode no longer loads the whole contract state to read
one slot out of it. ContractSnapshot pulls the entire `data` JSONB — every
storage slot the contract owns — which for a contract holding a large vector
store is a substantial fetch and deserialize on a call that batch tooling
polls hard. The slot is now extracted in SQL. Postgres still detoasts the
JSONB server-side, so this narrows transfer and parse cost rather than
eliminating the read; legacy and undeployed row shapes defer to the original
path to keep their error semantics intact.
The Lua script gains a test that actually executes it under a stubbed
redis.call. It was previously covered only through mocked evalsha, so an
error in the script body would have surfaced first in production, on every
/api request.
* feat(rate-limit): accept API keys in the URL path (#1742)
Keys could only be supplied as an X-API-Key header, which makes Studio
unusable from most of the EVM toolchain. MetaMask's "Add network" takes a
URL and has no header field at all; the same is true of foundry.toml,
--rpc-url, and most viem/ethers setups. Every major provider puts the key in
the path for this reason (Alchemy /v2/<key>, Infura /v3/<key>).
This is not hypothetical: an integrator on a shared Studio key told us they
would have to stand up a backend proxy "for them to be able to use it
properly as most of the use comes from testing deployments through
metamask". That proxy's whole job would be turning a URL into a header, and
a shared proxy is also what pushes teams onto one shared key instead of one
key per person.
POST /api/{api_key} now routes to the same JSON-RPC handler, with the header
still accepted for server-side callers. The path wins when both are present,
since it is what the caller typed rather than something an intermediary may
have injected. Only `glk_`-prefixed segments count as keys: any other
segment falls through to anonymous rather than erroring, while a mistyped
real key is still treated as a key so it fails loudly as invalid instead of
silently dropping to anonymous limits.
The middleware's path gate now covers every path the route can match, not
just key-shaped ones. A path served by the route but missed by the gate
would be an unlimited, unauthenticated RPC endpoint, so a test pins that
correspondence.
Keys in URLs get written down in places headers are not, and two of those
were ours: uvicorn access logging is on, and Sentry runs with
send_default_pii and full trace sampling, so request URLs leave the process
on every transaction, not only on errors. api_key_redaction scrubs keys from
both — a logging filter covering uvicorn's args-based access line, and
before_send/before_send_transaction hooks that walk the whole event rather
than known fields. Without this, path keys would be a downgrade on
header-only rather than an improvement.
* feat(rate-limit): classify the v0.123-only RPC methods
The ported allowlist predates three methods that exist only on this branch.
sim_getFeeConfig and gen_getTransactionStatusDetails are cheap reads — a
config lookup and a transactions_processor read respectively, neither
touching the GenVM.
sim_estimateTransactionFees is not. It takes validators_manager and
genvm_manager and runs the contract through sim_call to measure fees, so it
costs a full execution. Worth flagging because eth_estimateGas *is*
allowlisted — that one returns a hardcoded constant. The names are nearly
identical and the costs are not, which is the same shape of trap as eth_call
and the two schema methods already documented here.
* chore(genvm): repin to the v1 gl_call ABI and re-vendor origin ⬆️ (#1744)
Squashed: the v0.3.0-rc7 runner-hash bump and the gl_call v1 repin with the
re-vendored backend/node/genvm/origin/ mirror.
* feat(genvm): vendor leader public data codec (#1747)
* fix(fees): align Studio with Consensus v0.6 economics (#1748)
* fix(fees): align Studio with consensus v0.6 economics
Mirror cap-priced round quotes, configured ladder semantics, appeal profit reserves, exact 2.5x rewards, typed appeal funding, immutable top-up schedules, live appeal overlay pricing, latest appeal ABIs, decision guards, and surplus refunds. Expose the canonical quote over RPC so cross-stack tests no longer maintain a second Studio formula.
* test(fees): derive settlement deposit from quote
* chore(genvm): replace dead source pin
* test(fees): cover timeout rotation multiplier
* fix(fees): align studio with consensus v0.6
* fix(hardhat): restore reproducible deployment profile
* fix(consensus): match v0.6 round lifecycle
* fix(consensus): bind appeals to exact decisions
* feat(rpc): expose v0.6 decision lifecycle
* fix(fees): complete v0.6 runtime parity
* fix(consensus): pin appeal selection authority
* fix(consensus): make appeal admission failure value-safe
* fix(consensus): preserve paid appeals across worker failures
* fix(consensus): make internal message effects replay-safe
* fix(consensus): reject unregistered internal recipients
* fix(consensus): recover message effects atomically
* fix(rpc): keep pre-fee consensus entrypoints decodable
* fix(rpc): serve a name-unique consensus ABI
* fix(consensus): treat a rollup-free deployment as a valid emission path
* chore(genvm): repin to the merged leader-data commit on v0.6-dev
The pin inherited from v0.123-dev references the pre-merge feature branch
feat/leader-data-to-genvm, whose branch was deleted after squash-merge; the
commit is unreachable from any ref, so the source build's clone cannot read
it (fatal: unable to read tree). The same work is v0.6-dev head 3c96be6de
(feat(manager): carry opaque leader public data, #29), which also contains
the previous pin d07f21da. v0.123-dev itself needs the same repin.
* fix(consensus): carry the parent rotation schedule onto claims
* fix(consensus): bind message effects to protocol recipients
* fix(consensus): align child admission with v0.6
* fix(consensus): complete v0.6 studio parity
* fix(consensus): report exact creation slots
* fix(fees): close remaining consensus handoff gaps
Pin the complete activation policy and funding overlay, preserve pool-aware contribution and external remainder ownership, reject invalid allocation and contribution states, and refund activated child value to its actual funder. Honor exact initial committees and fee-plan-sized leader replays, including receipt-less Undetermined finalization when capacity is unavailable. Add the shared simulator quote oracle plus focused lifecycle, settlement, and PostgreSQL atomicity coverage.
* fix(rpc): decode submissions without a rollup connection
* fix(tooling): qualify the v0.6 fee train
Align Studio lifecycle/status RPCs with the node v0.6 consumer interface, pin the exact fee-aware SDK and test-harness revisions, and make integration/load callers submit the canonical SDK fee estimate. Preserve legacy status-string compatibility for existing Studio consumers.
* fix(consensus): close latest v0.6 handoff gaps
* fix(consensus): unwind descendant appeals on rewind
* fix(ghost): scope bundled factory salts by sender
* test(ghost): cover parent-scoped salt reuse
* fix(fees): derive exact transaction message allocations
* test(integration): submit canonical fee envelopes
* fix(studio): close v0.123 correctness gaps
* fix(rpc): hide unexpected server details
* fix(studio): let fee estimation measure full execution
* fix(fees): align Studio with GenVM fee buckets
Treat GenVM v0.3 bucket 0 as the shared execution reservoir, bucket 1 as message fees, and buckets 2/3 as byte counters. Reconstruct the exact GenVM receipt meter to derive Consensus storage/event-write fees, and size recommendations for both chargeable execution and the enforced shared reservoir.
* fix(fees): use canonical allocation wildcard
* chore(deps): pin corrected fee SDK heads
* chore(deps): repin restacked fee SDK heads
* fix(fees): match internal deploy allocations
* fix(consensus): bind salted children to GenVM addresses
* fix(fees): fund discovered child execution
* fix(rpc): expose mined envelope reverts
* chore(deps): repin receipt-aware JS SDK
* chore(deps): repin topup-safe JS SDK
* fix(fees): align envelope and shared allocations
* chore(deps): repin message event SDK
* fix(validators): replace pools atomically
* test(fees): lock fresh appeal capacity
Cover the two topology boundaries behind validator appeals: a five-validator pool is exhausted by the initial committee, while twelve identities provide the full fresh seven-seat first jury and preserve the exact Consensus quote.
* fix(release): harden studio parity boundaries
* test(ci): isolate fee default regression
* test(ci): fund gltest default submissions
* test(ci): fund gltest write defaults
gltest owns separate fee argument adapters for deploys and writes. Bridge both so their public None defaults are treated as omitted by the fee-aware integration fixture, while explicit direct SDK fees=None stays gasless. Add a dependency-free regression covering both entry points.
---------
Co-authored-by: kirilaa <kirilantevski@gmail.com>
* feat(release): support Studio prereleases (#1751)
Add a preview-only Studio RC release path, protect Docker latest, and display the deployed version.
* fix(release): default optional GenVM image refs (#1752)
Release image builds do not pass GenVM overrides. Define those optional arguments as empty in both Docker stages so nounset shells can select the checked-in GenVM pin, and cover the no-build-args workflow path with a fast parser regression.
* fix(release): pin GenVM RC artifacts (#1753)
Use the immutable GenVM v0.6.0-rc3 release for distributed backend image builds. Add a fast policy regression so a source branch/SHA cannot reach the release workflow without its required Nix closure.
* fix(release): classify default GenVM tags (#1754)
* fix: align local Studio chain defaults (#1756)
* fix: normalize faucet funding quantities (#1758)
* fix: normalize faucet funding quantities
* test: use active chain id for ghost address
* fix(rpc): commit funding credit before fund_account returns (#1760)
* fix(ci): repin genlayer-py to a compatible version
* feat(genvm): wire named fee accounting ⬆️ (#1750)
* fix(genvm): restore legacy v0.2 contract compatibility (#1762)
Route legacy v0.2 contracts to the retained executor and preserve executor-specific code layouts through schema, deploy simulations, finalized execution, reads, and upgrades.
Validated combined candidate tree 7150e217a417af59fd1d1852f5f9573edad4ae9e over v0.123-dev@0737a52de4be3db0ea33336d99f68b5e3e14ce40. Full E2E 35199189091: 174 passed, 107/107 features covered, scope=all and stack=all. Combined backend unit suite: 1,631 passed.
Single-PR landing under the owner's explicit admin-merge authorization; no cross-repository PR dependencies. Preview release only.
* fix(fees): align Studio execution budget floor with Consensus (#1783)
Price the proposed receipt wrapper plus the 64-byte terminal reserve when quoting and validating the minimum execution budget. Keep the legacy 512-byte receipt estimator separate. Cover the canonical v0.6 quote, alternate wrapper sizes, exact admission boundary, and locked top-up formula.
Validated with 1,635 Studio unit tests and repository pre-commit hooks.
* fix(genvm): forward flat fee allocation inputs (#1774)
* fix(genvm): forward flat fee allocation inputs 🐛
Re-vendor the fee allocation types and replace recursive children with
checked direct-child budgets and host-encoded FlatArrays subtrees.
Preserve committed fee bytes, ordering, roots and original allowances.
Pin GenVM v0.6.0-rc7, the release cut from the fee-allocation branch.
* test(genvm): pin the reroute test to the rc9 executor line ✅
GenVM v0.6.0-rc7 ships executor v0.3.0-rc9; the missing v0.3.0-rc7 directory left the pinned deploy stuck in PROPOSING
* chore(docker): use the runner-baked webdriver 0.0.12 (#1786)
* feat(fees): emulate descendant grants of balance-funded messages (#1785)
* chore(genvm): pin descendant-grants genvm and refresh runner hashes ⬆️
* feat(fees): emulate descendant grants of balance-funded messages ✨🐛
Studio also read the emission budget from `declaredBudget`, a key the manager never sends (`message_fee`), so every balance-funded child was declared with budget 0.
* chore(genvm): pin genvm v0.6.0-rc9 and refresh runner hashes ⬆️
* fix(fees): preserve descendant funding and grant restrictions
Inherit the parent fee depositor and admitted grant rules when materializing descendants. Preserve metering for zero-budget grant descendants and wait through all triggered transaction levels in the integration test.
Validation: 1669 unit tests passed, 7 skipped; Black and diff checks passed. Live integration was not rerun.
* refactor(fees): treat omitted descendant grants as closed
Remove the legacy grant flag and its permissive fee validation and rotation rules. Omitted grants now use the same validation and accounting as explicit Closed grants, with surplus retained in primary escrow.
Validation: 1674 unit tests passed, 7 skipped; Black and diff checks passed. Independent review found no actionable issues.
* test(fees): run the descendant wait test with the integration suite
It imports the integration module, which needs gltest; the unit job does not install it
Claude-Session: https://claude.ai/code/session_01N97Hvs14ckrioBoYJTD4Te
* fix(docker): default the nix netrc secret to an empty file in the repo
buildx bake refuses to read /dev/null without --allow=fs.read=/dev/null, which compose cannot pass, so builds with buildx 0.38 failed
Claude-Session: https://claude.ai/code/session_01N97Hvs14ckrioBoYJTD4Te
* fix(validators): roll back the registry session on failure (#1795)
The RPC process holds one validators session for its whole lifetime, and
every gen_call, sim_call and sim_estimateTransactionFees reads the registry
through it (Manager.snapshot). When the Postgres connection dropped
mid-query, nothing rolled that session back, so every later execution
failed with PendingRollbackError ("Can't reconnect until invalid
transaction is rolled back") until the pod restarted. Plain RPC reads use
per-request sessions and kept working, which is the pattern in #1790.
Registry and interceptor methods now roll the session back on any
exception before re-raising. Success paths are unchanged.
The new db-sqlalchemy test terminates the session's backend with
pg_terminate_backend; without the fix the next read raises the exact
error from the report.
Fixes #1790
* fix(genvm): serve latest-finalized storage reads from finalized state 🐛 (#1792)
* chore(release): v0.123 RC fixes (#1765, #1715, #1773, #1789) (#1797)
Lands four reviewed fork PRs together after a full E2E run on this exact head:
- #1765: redact validator private keys from call receipts and explorer payloads
- #1715: bind postgres, redis and ollama host ports to loopback
- #1773: normalize addresses in raw-SQL balance operations
- #1789: normalize the address in sim_fundAccount
Co-authored-by: Kanan <93033289+kriss39@users.noreply.github.com>
Co-authored-by: Starling <82058629+Starling-spell@users.noreply.github.com>
Co-authored-by: Alicepoltora <Alicepoltora@users.noreply.github.com>
* fix(examples): ask LlmErc20 for single-brace JSON (#1798)
The response-format block lives in a plain (non f-) string, so the
doubled braces reached the LLM literally. A real model follows the
template and answers `{{...}}`, which json.loads rejects. This made
test_llm_erc20 fail whenever the mock did not apply and the call fell
through to the real provider (CI runs on #1784, 2026-09-10).
Co-authored-by: Claude <noreply@anthropic.com>
* fix(fees): reserve parent appeal capacity in Studio message allocations (#1784)
* fix(genvm): forward flat fee allocation inputs 🐛
Re-vendor the fee allocation types and replace recursive children with
checked direct-child budgets and host-encoded FlatArrays subtrees.
Preserve committed fee bytes, ordering, roots and original allowances.
Pin GenVM v0.6.0-rc7, the release cut from the fee-allocation branch.
* test(genvm): pin the reroute test to the rc9 executor line ✅
GenVM v0.6.0-rc7 ships executor v0.3.0-rc9; the missing v0.3.0-rc7 directory left the pinned deploy stuck in PROPOSING
* fix(fees): reserve parent appeal capacity in Studio allocations
Price each on-acceptance internal allocation using its emitting parent’s prepaid appeal horizon and multiply the complete child requirement, including direct child budgets. Carry the root appeal count from the transaction or child fee distribution.
Fund every discovered acceptance emission opportunity while keeping an inferred per-message declared budget to one child primary plus its direct subtree. Cover the 360/1440 boundary, nested parents, finalization, discovery, and child accounting.
---------
Co-authored-by: kp2pml30 <kp2pml30@gmail.com>
Co-authored-by: Daniel Rojas <danielrojasc888@gmail.com>
* fix(consensus): finalize transaction-local accepted state (#1796)
* fix(consensus): finalize transaction-local accepted state
Persist each successful acceptance's post-state alongside its rollback snapshot, then reload only that post-state during finalization. Verify receipt hashes and preserve agreed decisions when legacy state cannot be recovered safely, rather than promoting a younger accepted head or canceling the older transaction.
Cover overlap, deployments, failed and successful appeals, recomputation, empty state, execution errors, and legacy rows through the real PostgreSQL worker path. No schema migration is required. Rollout must drain or explicitly recover pre-fix in-flight decisions that lack verifiable state.
Validation: 1690 unit tests passed (repository-recommended RPC manager exclusion); 244 database tests passed, 1 existing xfail; changed-file pre-commit checks passed. Full stack E2E not run. Local branch only pending release-plan confirmation.
* fix(consensus): skip backed-off finalization heads without reordering contracts
A finalization head in retry backoff was claimed and released every poll,
so the oldest stuck head starved every other contract's finalization.
claim_next_finalization now takes the worker's backed-off hashes and skips
contracts whose head (or an older non-terminal tx) is backing off, while
the existing ordering guard still blocks younger same-contract txs.
Retry tracking is cleared once a finalization succeeds.
* fix(consensus): recover legacy finalization state from successor pre-states
Rows accepted before transaction-local finalization state was persisted
can only finalize when the moving contract head still hashes to their
receipt. Once a successor is accepted the head moves and the row is stuck.
A successor's saved pre-state is exactly the predecessor's post-state
when nothing intervened, so after the existing sources fail the resolver
now tries the transaction's own pre-state (unchanged-state writes) and up
to three successor pre-states, accepting only one whose hash matches the
receipt's contract_state_hash. Anything else still raises
FinalizationStateError without canceling.
Adds a read-only finalization_legacy_report CLI that classifies pending
legacy rows with the same resolver before deploy.
* fix(consensus): finalize decided predecessors before a code upgrade applies
A type 3 upgrade used to swap the code slot while earlier decided
transactions on the same contract were still inside their appeal window.
Those rows then finalized, or were re-executed by an appeal, against code
they were never decided on, and their paid appeals stayed open across the
code change.
The upgrade now closes the contract's earlier lifecycle before touching code:
- It defers while any other row on the contract holds a lease or is
mid-consensus, so it never races live work (UPGRADE_DEFER_SECONDS).
- Every ACCEPTED+SUCCESS predecessor must resolve a verified finalization
state up front. If one cannot, the upgrade is canceled with
predecessor_finalization_state and nothing is changed.
- Each decided predecessor (ACCEPTED, UNDETERMINED, *_TIMEOUT) is
force-finalized through the normal path: acceptance delivery repair,
messages, fee settlement and value refunds. A pending paid appeal is
refunded first via abort_pending_appeal_admission, which only aborts an
admission with no recorded outcome.
- Progress (closed rows, refunds) is persisted on the upgrade row in the
same commit as each refund, and that write also renews the lease. A retry
skips rows that are already final and can never refund twice.
- The code is applied last, in one commit with the FINALIZED status, after
rechecking ownership and that no predecessor reopened.
- Never-started PENDING rows stay queued and run on the new code.
Validation and contract errors cancel the upgrade as before. Transient
failures go through the generic retry path, and rows that are already
finalized stay final.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(consensus): upgrades permanently close the appeal and rewind lifecycle
Force-finalizing predecessors only holds if nothing can reopen them
afterwards. Before this change, an appeal could still be admitted while
the upgrade was queued. A successful appeal's rewind also reset every
newer row on the contract to PENDING, including FINALIZED and CANCELED
rows and the upgrade itself. That replayed an applied upgrade and
resurrected terminal transactions.
- Appeal admission is rejected with CanNotAppeal when a PENDING or
FINALIZED upgrade is queued after the decision. The check runs in the
RPC precheck and again under the row lock in admit_transaction_appeal.
- Rewinds only reset rows that can still be replayed
(get_rewindable_newer_transactions). Terminal rows and upgrades are
never reset, and rollback_transactions guards this again in memory.
Upgrades applied before this change could overtake decided rows that are
still open. When such an older row finalizes, or a validator appeal on it
restores its pre-state, the code of the latest FINALIZED upgrade after it
is kept (with_later_upgrade_code). Without this, the old code would come
back. The legacy finalization report now also lists these rows as
"interleaved upgrade (<hash>)" so operators can see them before deploying.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(consensus): keep untyped legacy rows rewindable
`type != UPGRADE_CONTRACT` evaluates to NULL for rows with no type, so
the rewind query silently dropped them. Treat a NULL type as rewindable.
Validation: unit 1731 passed (known env-only test_studio_fees failure),
PostgreSQL suite 268 passed, 1 xfailed.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Daniel Rojas <danielrojasc888@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
* fix(rpc): keep database error details out of RPC responses (#1801)
An unexpected exception in a JSON-RPC handler was returned to the client
as str(exc). For SQLAlchemy errors that text carries the SQL statement
and all bound values, so a failed INSERT into validators could return a
validator private key. sim_createValidator was also the only validator
mutator without the hosted-mode guard.
- Gate create_validator with check_forbidden_method_in_hosted_studio,
like the other validator mutators; the hosted UI already hides it.
- Set hide_parameters=True on the RPC and consensus worker engines so
no exception text (responses, logs, persisted receipt errors) includes
bound values.
- Return "Internal database error" for SQLAlchemy errors in the RPC
catch-all; other unexpected errors keep their message.
Tests cover each part and fail when the corresponding change is reverted.
* fix(genvm): keep v0.2 contracts able to deploy their v0.2 children (#1802)
* fix(genvm): route legacy-runner contracts to the retained v0.2 executor
The manager picks an executor line by major version only, so every
major-0 contract that is not explicitly `# v0.2.x` resolves to the newest
line (v0.3). That line ships only its own runners and rejects older ones
as `invalid_contract runner malformed`. Contracts with a `# v0.1.x` header,
no version line, a Seq header, or a zip runner.json that depend on v0.2
runners could therefore not be deployed on v0.123, and factories could not
deploy such children (child deploys carry no executor pin).
Also select the retained line when the code's runner spec (parsed like
GenVM: zip runner.json, else the leading comment block after an optional
version line) declares a runner that appears in an installed
`$GENVMROOT/executor/v0.2.*/data/all.json` and in no newer line.
Contracts that existed before the multi-line upgrade are already pinned by
migration c1d2e3f4a5b6, so no data …
Delivery context
Target:
v0.123-dev. This is an urgent, independently releasable compatibility fix for the Studio Next / Studio Dev preview deployment. It does not promote the standing stable-release gate #1670 and has no cross-repository dependency.Landing cut: this PR only, head
7c0bc132dbee0ee0731ebb2a872519303f2d6847overv0.123-dev@9d414f99ee69b2de414a3014dea49e62f20382de. The named fee-bucket work in #1750 / genvm-manager#31 is deliberately excluded: it is a separate protocol change and is not needed for this fix. Keep this cut frozen during E2E.What
sim_config.gen_calland fee-estimate simulations agree with consensus.Why
Studio's default major-0 selection chooses the current v0.3 executor, but retained v0.2 contracts use a different runner format. Their unchanged, stable runner hashes therefore fail before contract execution.
Routing alone is insufficient: v0.2 stores CODE at root offset 1 and LOCKED_SLOTS at 2, whereas v0.3 stores CODE at 2. Putting schema code into the modern slot makes the legacy executor interpret the code-length prefix as a lock-list count. Code reads and finalized bootstrap also need to use the legacy layout.
Testing done
test_rpc_endpoint_manager.pyexclusion; one existing websockets deprecation warning).pre-commit run --all-filesin an isolated checkout: all general/backend hooks passed; frontend ESLint/Prettier could not start because frontend dependencies are not installed in that scratch checkout. No frontend files changed.tests/integration/test_deploy_reroute_to.py; full E2E is the remaining landing gate.Decisions made
Checks
Reviewing tips
The first-line selector is in
backend/node/genvm/executor_selection.py. Review it together with the ABI-specific code-slot handling and the accepted-state registration regression, since a routing-only fix still fails.User-facing release notes
Restores schema extraction and ordinary deployment of unchanged legacy v0.2 contracts on Studio preview, and retains the correct executor/storage layout for later reads and writes.