Repository navigation
ci: optimize integration test pipeline (~4-5 min faster) - #1440
Conversation
- Cache GenVM precompile artifacts across CI runs (keyed on GENVM_TAG) - Add docker-compose.ci.yml override to mount host cache into containers - Enable pip cache on setup-python (cache-dependency-path for 3 req files) - Move Python setup + pip install before Docker build (overlap I/O) - Combine 3 pip install commands into 1 (single dependency resolution) - Bump test parallelism from -n 4 to -n 8 (I/O-bound tests benefit) - Reduce health check poll interval from 5s to 2s - Add mkdir -p safety in entrypoints for empty cache on first run
The full CRUD suite ran 5 iterations of create/delete, each triggering a GenVM LLM module restart (~14s per call, ~280s total). Replaced with a single create-delete smoke test (~30s) that still exercises the GenVM restart path. Exhaustive CRUD coverage lives in tests/db-sqlalchemy/validators_registry_test.py (<1s, no GenVM overhead).
📝 WalkthroughWalkthroughAdds GenVM precompile caching into CI and mounts it into compose services; moves Python setup earlier in CI; ensures cache-marker directories exist in entrypoints; introduces shared decoded-value & contract-snapshot caches, state‑proxy metrics, and validator execution timeout handling in backend; replaces a heavy validator CRUD integration test with a lighter create/delete smoke test and adds related unit tests and benchmarks. Changes
Sequence Diagram(s)sequenceDiagram
participant CI as CI Workflow
participant Py as Python Setup
participant Buildx as Docker Buildx
participant Cache as GenVM Cache Store
participant Compose as Docker Compose (CI)
participant Service as Containers (jsonrpc / consensus-worker)
participant GenVM as GenVM Host (in-container)
participant Tests as Integration Tests
CI->>Py: setup Python 3.12 & install deps (pip cache)
Py-->>CI: deps ready
CI->>Buildx: build images (extract GENVM_TAG)
Buildx->>Cache: restore genvm-precompile-<tag>-amd64
Cache-->>Buildx: provide precompile cache
CI->>Compose: run docker-compose (no-build) with GENVM_CACHE_DIR mounted
Compose->>Service: start services with /genvm-cache mount
Service->>GenVM: run precompilation (mkdir -p then touch marker)
GenVM-->>Service: precompile marker present
Service->>Compose: health probe success (curl)
Compose->>Tests: run integration tests (parallel 8, validator smoke)
Tests-->>CI: test results
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
requirements.txt pins requests==2.32.5 while backend/requirements.txt pins requests==2.32.3. Combined into a single pip install, pip correctly rejects the conflict. Revert to sequential installs (last writer wins) to match existing behavior. The version mismatch should be fixed in a separate PR.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In @.github/workflows/backend_integration_tests_pr.yml:
- Around line 130-141: The health-check loop's wall-clock timeout was
inadvertently reduced because the poll interval (sleep 2) changed but timeout
variable remained 60 (iterations): update the loop to restore the original ~300s
budget by setting timeout=150 (or compute timeout based on desired
seconds/interval) so the loop using counter, timeout, and sleep 2 will wait ~300
seconds again; alternatively, rename timeout to max_attempts and add a comment
clarifying it counts iterations to avoid future confusion.
| timeout=60 | ||
| counter=0 | ||
| while [[ "$counter" -lt "$timeout" ]]; do | ||
| if curl -X POST -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","method":"ping","params":[],"id":1}' http://0.0.0.0:4000/api | grep -q "OK"; then | ||
| if curl -sS -X POST -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","method":"ping","params":[],"id":1}' http://0.0.0.0:4000/api 2>/dev/null | grep -q "OK"; then | ||
| echo "RPC server is up!" | ||
| break | ||
| else | ||
| echo "Waiting for RPC server... ($counter/$timeout)" | ||
| sleep 5 | ||
| sleep 2 | ||
| counter=$((counter+1)) | ||
| fi | ||
| done |
There was a problem hiding this comment.
Effective health-check timeout halved from ~300s to ~120s.
The counter counts iterations (not seconds). With the poll interval changed from 5s to 2s but timeout=60 unchanged, the maximum wait dropped from 60×5=300s to 60×2=120s. If that's intentional, great — but if services occasionally need more than 2 minutes to start, this will cause spurious failures.
Consider either bumping the iteration count to preserve the original wall-clock budget, or renaming the variable to clarify it represents iterations:
Option: restore ~300s wall-clock timeout
- timeout=60
+ timeout=150📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| timeout=60 | |
| counter=0 | |
| while [[ "$counter" -lt "$timeout" ]]; do | |
| if curl -X POST -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","method":"ping","params":[],"id":1}' http://0.0.0.0:4000/api | grep -q "OK"; then | |
| if curl -sS -X POST -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","method":"ping","params":[],"id":1}' http://0.0.0.0:4000/api 2>/dev/null | grep -q "OK"; then | |
| echo "RPC server is up!" | |
| break | |
| else | |
| echo "Waiting for RPC server... ($counter/$timeout)" | |
| sleep 5 | |
| sleep 2 | |
| counter=$((counter+1)) | |
| fi | |
| done | |
| timeout=150 | |
| counter=0 | |
| while [[ "$counter" -lt "$timeout" ]]; do | |
| if curl -sS -X POST -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","method":"ping","params":[],"id":1}' http://0.0.0.0:4000/api 2>/dev/null | grep -q "OK"; then | |
| echo "RPC server is up!" | |
| break | |
| else | |
| echo "Waiting for RPC server... ($counter/$timeout)" | |
| sleep 2 | |
| counter=$((counter+1)) | |
| fi | |
| done |
🤖 Prompt for AI Agents
In @.github/workflows/backend_integration_tests_pr.yml around lines 130 - 141,
The health-check loop's wall-clock timeout was inadvertently reduced because the
poll interval (sleep 2) changed but timeout variable remained 60 (iterations):
update the loop to restore the original ~300s budget by setting timeout=150 (or
compute timeout based on desired seconds/interval) so the loop using counter,
timeout, and sleep 2 will wait ~300 seconds again; alternatively, rename timeout
to max_attempts and add a comment clarifying it counts iterations to avoid
future confusion.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@backend/node/genvm/base.py`:
- Around line 437-447: The retry logic risks copying a mutated state_proxy
because the original host_args[state_proxy] can be altered during the first
attempt; fix by capturing a pristine snapshot up-front and using that for
retries instead of deepcopying the possibly-mutated proxy. Specifically, before
the first execution attempt, capture original_state_proxy =
host_args.get("state_proxy") and, if it has snapshot_factory, create and store a
pristine snapshot (via snapshot_factory/_copy_state_proxy) to use when building
fresh_args on retry; update the fresh_args construction (the block using
retry_count, host_args, key == "state_proxy", snapshot_factory, and
_copy_state_proxy) to prefer the preserved pristine snapshot for retries rather
than deepcopying the current host_args[state_proxy].
🧹 Nitpick comments (2)
backend/consensus/base.py (1)
2929-2938: Don’t swallow unexpected exceptions during cancel.Catching broad
Exceptionhere hides failures from validator execution; it’s safer to only silence cancellation/timeouts.Proposed diff
- except ( - asyncio.CancelledError, - asyncio.TimeoutError, - Exception, - ): - pass + except (asyncio.CancelledError, asyncio.TimeoutError): + passtests/unit/consensus/test_validator_exec_timeout.py (1)
101-106: Timing assertion may be flaky under CI load.The hung validator's
asyncio.waittimeout is 0.01 s, plus up to 0.1 s for the cancel grace period, leaving only ~0.04 s of headroom before the 0.15 s cap. Under heavy CI load or slow event-loop scheduling this could intermittently fail.Consider bumping the threshold (e.g.,
< 0.5) or asserting a relative relationship (elapsed significantly less than the 0.2 s hung sleep) to avoid flakiness without weakening the test's intent.
| def _build_timeout_receipt(validator_dict: dict) -> Receipt: | ||
| timeout_ms = int(validator_exec_timeout_seconds * 1000) | ||
| return Receipt( | ||
| result=bytes([ResultCode.VM_ERROR]) + b"timeout", | ||
| calldata=b"", | ||
| gas_used=0, | ||
| mode=ExecutionMode.VALIDATOR, | ||
| contract_state={}, | ||
| node_config=validator_dict, | ||
| eq_outputs={}, | ||
| execution_result=ExecutionResultStatus.ERROR, | ||
| vote=None, | ||
| genvm_result={ | ||
| "stdout": "", | ||
| "stderr": ( | ||
| "Validator execution exceeded " | ||
| f"{validator_exec_timeout_seconds:.3f}s" | ||
| ), | ||
| "error_code": "CONSENSUS_VALIDATOR_EXEC_TIMEOUT", | ||
| "raw_error": { | ||
| "causes": ["VALIDATOR_EXEC_TIMEOUT"], | ||
| # Mark as fatal so replacement validators are attempted. | ||
| "fatal": True, | ||
| }, | ||
| }, | ||
| processing_time=timeout_ms, | ||
| ) | ||
|
|
There was a problem hiding this comment.
Timeout-triggered replacements should still use VRF selection.
By marking timeouts as fatal, this path now triggers validator replacement; ensure the replacement selection is VRF-based (instead of pool-order pop) to match consensus requirements.
As per coding guidelines, Implement validator rotation using VRF (Verifiable Random Function) in the consensus system.
| # Avoid expensive state deep-copy on the first attempt. We only need | ||
| # a clean copy when retrying after a failed execution attempt. | ||
| if retry_count == 0: | ||
| fresh_args = dict(host_args) | ||
| else: | ||
| fresh_args = {} | ||
| for key, value in host_args.items(): | ||
| if key == "state_proxy" and hasattr(value, "snapshot_factory"): | ||
| fresh_args[key] = await _copy_state_proxy(value) | ||
| else: | ||
| fresh_args[key] = copy.deepcopy(value) |
There was a problem hiding this comment.
Retry now risks using a dirty state_proxy after a failed attempt.
The first attempt passes the original proxy; if GenVM mutates storage before an exception, subsequent retries deepcopy the already-mutated proxy from host_args, breaking the “clean retry” guarantee. Consider keeping a pristine snapshot for retries (or reconstructing via snapshot_factory) rather than copying the possibly-mutated proxy.
🤖 Prompt for AI Agents
In `@backend/node/genvm/base.py` around lines 437 - 447, The retry logic risks
copying a mutated state_proxy because the original host_args[state_proxy] can be
altered during the first attempt; fix by capturing a pristine snapshot up-front
and using that for retries instead of deepcopying the possibly-mutated proxy.
Specifically, before the first execution attempt, capture original_state_proxy =
host_args.get("state_proxy") and, if it has snapshot_factory, create and store a
pristine snapshot (via snapshot_factory/_copy_state_proxy) to use when building
fresh_args on retry; update the fresh_args construction (the block using
retry_count, host_args, key == "state_proxy", snapshot_factory, and
_copy_state_proxy) to prefer the preserved pristine snapshot for retries rather
than deepcopying the current host_args[state_proxy].
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Fix all issues with AI agents
In `@backend/consensus/base.py`:
- Around line 2925-2944: The current cancellation block around exec_task created
from node.exec_transaction swallows all exceptions via a broad except Exception;
narrow this to only handle asyncio.CancelledError and asyncio.TimeoutError (the
expected outcomes) and let any other exceptions propagate (or re-raise/log them)
so real errors aren't suppressed; locate the block referencing exec_task,
validator_exec_timeout_seconds and node.exec_transaction and replace the generic
except Exception with explicit except clauses for CancelledError and
TimeoutError, handling those silently while ensuring other exceptions are not
caught.
In `@config-overrides.example/genvm-module-web.yaml`:
- Around line 13-17: The signer_url template value uses the wrong interpolation
syntax; locate the signer_url entry (key: signer_url) where its value is
currently "#{signerUrl}" and change it to use the repository's standard
"${signerUrl}" interpolation so it matches other YAML configs.
In `@scripts/benchmarks/snapshot_bench.py`:
- Around line 16-135: Add explicit type hints to the benchmark helpers: annotate
make_snapshot(addr: str, slots_per_contract: int, value_size: int) ->
SimpleNamespace; make_view(snapshot_cls: Type, primary_snapshot:
SimpleNamespace, snapshot_factory: Callable[[str], Any], shared_cache:
Optional[Mapping[Any, Any]] = None) -> Any (or the snapshot instance type);
build_slots(slots_per_contract: int) -> List[bytes]; run_scenario(scenario:
Dict[str, Any], snapshot_cls: Type, address_cls: Type, executions: int,
use_shared_cache: bool) -> Dict[str, Any]; and main() -> None; and add imports
for typing symbols used (Type, Callable, Any, Optional, Mapping, List, Dict) so
static checkers accept the annotations. Ensure default values/signatures remain
unchanged while adding the return types.
In `@scripts/benchmarks/validator_batch_bench.py`:
- Around line 97-112: The loop can raise ZeroDivisionError when cross_addresses
or slots are empty; validate inputs before the loop and guard modulo/division
uses: ensure that if cross_reads > 0 then cross_addresses is not empty (or
reduce/skip cross_reads), and ensure slots (from slots_per_contract) is
non-empty before using len(slots) in sidx calculations; update the code around
variables cross_addresses, slots, cross_reads, local_reads, cross_done and the
for-loop that computes sidx/cidx to either validate/raise a clear error earlier
or branch to skip cross reads and local reads when their corresponding
address/slot lists are empty so no modulo by zero occurs.
In `@tests/direct/storage_read_bench.py`:
- Around line 1-2: Replace the star import in this test module with explicit
imports to remove F403/F405 lint flags and clarify dependencies: import the
specific symbols used (gl, u256, DynArray) from genlayer (i.e., replace "from
genlayer import *" with an explicit import that lists gl, u256, DynArray) so the
test only exposes the needed names.
🧹 Nitpick comments (7)
tests/integration/icontracts/contracts/genvm_smoke_v1.py (1)
15-20: Double.strip()on prompt result.
ask_llm()already strips the result (line 18), and the return value ofstrict_eqis stripped again (line 20). One of them is redundant.Proposed fix — strip in one place only
def ask_llm() -> str: return gl.nondet.exec_prompt( "Respond with exactly the two characters OK and nothing else." - ).strip() + ) - self.prompt_result = gl.eq_principle.strict_eq(ask_llm).strip() + self.prompt_result = gl.eq_principle.strict_eq(ask_llm).strip()Or keep it only inside
ask_llm— either way, one call is sufficient.tests/direct/test_genvm_smoke_direct.py (1)
3-3: Unusedimport pytest.
pytestis imported but never used in this file.Remove unused import
-import pytesttests/direct/contracts/error_llm_contract_direct.py (1)
22-41: Remove extraneousfprefix from the prompt string.The f-string on line 23 has no interpolation placeholders — the
{{/}}are just producing literal braces. A plain string with{/}is clearer and avoids the Ruff F541 warning.Proposed fix
def test_execute_prompt(self): - prompt = f""" + prompt = """ What is 2+2? Respond using ONLY the following format: -{{ +{ "answer": int, -}} +} It is mandatory that you respond only using the JSON format above, nothing else. Don't include any other words or characters, your output must be only JSON without any formatting prefix or suffix.scripts/benchmarks/snapshot_bench.py (1)
26-31: Thread the shared contract snapshot cache intomake_view.
Right now the benchmark only exercisesshared_decoded_value_cache. Passingshared_contract_snapshot_cachewhen supported lets you measure the cross‑contract snapshot cache too (matchingvalidator_batch_bench.py).♻️ Suggested update
-def make_view(snapshot_cls, primary_snapshot, snapshot_factory, shared_cache): +def make_view( + snapshot_cls, + primary_snapshot, + snapshot_factory, + shared_decode_cache, + shared_snapshot_cache, +): sig = inspect.signature(snapshot_cls.__init__) kwargs = {"readonly": True} if "shared_decoded_value_cache" in sig.parameters: - kwargs["shared_decoded_value_cache"] = shared_cache + kwargs["shared_decoded_value_cache"] = shared_decode_cache + if "shared_contract_snapshot_cache" in sig.parameters: + kwargs["shared_contract_snapshot_cache"] = shared_snapshot_cache return snapshot_cls(primary_snapshot, snapshot_factory, **kwargs) - shared_cache = {} if use_shared_cache else None + shared_decode_cache = {} if use_shared_cache else None + shared_snapshot_cache = {} if use_shared_cache else None ... - view = make_view(snapshot_cls, primary_snapshot, snapshot_factory, shared_cache) + view = make_view( + snapshot_cls, + primary_snapshot, + snapshot_factory, + shared_decode_cache, + shared_snapshot_cache, + ) ... - "shared_cache_size": len(shared_cache) if shared_cache is not None else 0, + "shared_decode_cache_size": len(shared_decode_cache) + if shared_decode_cache is not None + else 0, + "shared_snapshot_cache_size": len(shared_snapshot_cache) + if shared_snapshot_cache is not None + else 0,Also applies to: 73-85, 125-128
tests/unit/consensus/test_helpers.py (1)
616-617: Unused parameters are intentional but could be clearer.Static analysis flags
shared_decoded_value_cacheandshared_contract_snapshot_cacheas unused (Ruff ARG001). Since this mock must match the productionNodeFactorysignature, this is expected. Consider prefixing with_to signal intent and silence the warning.Proposed fix
- shared_decoded_value_cache: Optional[dict[str, bytes]], - shared_contract_snapshot_cache: Optional[dict[str, ContractSnapshot]], + _shared_decoded_value_cache: Optional[dict[str, bytes]], + _shared_contract_snapshot_cache: Optional[dict[str, ContractSnapshot]],scripts/benchmarks/validator_batch_bench.py (2)
24-38:inspect.signatureintrospection is fragile — prefer explicit kwargs.If
_SnapshotView.__init__renames a parameter, this will silently skip passing the cache without any error. Since the benchmark already imports the concrete class, pass the kwargs directly.Proposed fix
def make_view( snapshot_cls, primary_snapshot, snapshot_factory, *, shared_decode_cache, shared_snapshot_cache, ): - sig = inspect.signature(snapshot_cls.__init__) - kwargs = {"readonly": True} - if "shared_decoded_value_cache" in sig.parameters: - kwargs["shared_decoded_value_cache"] = shared_decode_cache - if "shared_contract_snapshot_cache" in sig.parameters: - kwargs["shared_contract_snapshot_cache"] = shared_snapshot_cache - return snapshot_cls(primary_snapshot, snapshot_factory, **kwargs) + return snapshot_cls( + primary_snapshot, + snapshot_factory, + readonly=True, + shared_decoded_value_cache=shared_decode_cache, + shared_contract_snapshot_cache=shared_snapshot_cache, + )This also makes the
import inspecton line 4 unnecessary.
14-21: Missing type hints on public functions.
make_snapshot,run_mode, andmainlack type annotations. As per coding guidelines, "Include type hints in all Python code".Also applies to: 41-52, 129-157
| exec_task = asyncio.create_task( | ||
| node.exec_transaction(context.transaction) | ||
| ) | ||
| done, _ = await asyncio.wait( | ||
| {exec_task}, | ||
| timeout=validator_exec_timeout_seconds, | ||
| return_when=asyncio.FIRST_COMPLETED, | ||
| ) | ||
| if exec_task in done: | ||
| result = await exec_task | ||
| else: | ||
| exec_task.cancel() | ||
| try: | ||
| await asyncio.wait_for(exec_task, timeout=0.1) | ||
| except ( | ||
| asyncio.CancelledError, | ||
| asyncio.TimeoutError, | ||
| Exception, | ||
| ): | ||
| pass |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cd /tmp && find . -name "base.py" -path "*/consensus/*" 2>/dev/null | head -20Repository: genlayerlabs/genlayer-studio
Length of output: 54
🏁 Script executed:
git ls-files | grep -E "backend/consensus/base\.py"Repository: genlayerlabs/genlayer-studio
Length of output: 97
🏁 Script executed:
wc -l backend/consensus/base.pyRepository: genlayerlabs/genlayer-studio
Length of output: 102
🏁 Script executed:
sed -n '2920,2950p' backend/consensus/base.pyRepository: genlayerlabs/genlayer-studio
Length of output: 1588
🏁 Script executed:
head -30 backend/consensus/base.pyRepository: genlayerlabs/genlayer-studio
Length of output: 1019
🏁 Script executed:
grep -n "contextlib" backend/consensus/base.pyRepository: genlayerlabs/genlayer-studio
Length of output: 54
🏁 Script executed:
grep -n "import" backend/consensus/base.py | head -40Repository: genlayerlabs/genlayer-studio
Length of output: 1980
Narrow the exception handler to avoid swallowing unexpected errors.
The broad except Exception clause can hide real errors during task cancellation. The expected outcomes are cancellation or timeout; other exceptions should not be suppressed.
♻️ Suggested cleanup
+import contextlib
import os
import asyncio- try:
- await asyncio.wait_for(exec_task, timeout=0.1)
- except (
- asyncio.CancelledError,
- asyncio.TimeoutError,
- Exception,
- ):
- pass
+ with contextlib.suppress(
+ asyncio.CancelledError,
+ asyncio.TimeoutError,
+ ):
+ await asyncio.wait_for(exec_task, timeout=0.1)🧰 Tools
🪛 Ruff (0.15.0)
[error] 2939-2944: try-except-pass detected, consider logging the exception
(S110)
[warning] 2942-2942: Do not catch blind exception: Exception
(BLE001)
🤖 Prompt for AI Agents
In `@backend/consensus/base.py` around lines 2925 - 2944, The current cancellation
block around exec_task created from node.exec_transaction swallows all
exceptions via a broad except Exception; narrow this to only handle
asyncio.CancelledError and asyncio.TimeoutError (the expected outcomes) and let
any other exceptions propagate (or re-raise/log them) so real errors aren't
suppressed; locate the block referencing exec_task,
validator_exec_timeout_seconds and node.exec_transaction and replace the generic
except Exception with explicit except clauses for CancelledError and
TimeoutError, handling those silently while ensuring other exceptions are not
caught.
| lua_script_path: ${exeDir}/../config/genvm-web-default.lua | ||
| vm_count: 6 | ||
| lua_path: ${exeDir}/../lib/genvm-lua/?.lua | ||
| signer_url: "#{signerUrl}" | ||
| signer_headers: {} |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cat config-overrides.example/genvm-module-web.yamlRepository: genlayerlabs/genlayer-studio
Length of output: 1276
🏁 Script executed:
#!/bin/bash
# Check both template syntaxes in YAML files across the repo
echo "=== \${...} syntax in YAML files ==="
rg -n '\$\{[^}]+\}' --type yaml | head -30
echo ""
echo "=== #{...} syntax in YAML files ==="
rg -n '#\{[^}]+\}' --type yaml | head -30Repository: genlayerlabs/genlayer-studio
Length of output: 2226
Fix template syntax on line 16: use ${signerUrl} instead of #{signerUrl}.
The ${...} syntax is the standard interpolation pattern across all YAML configs in the repository (docker-compose.yml, vector.yaml, etc.). The #{...} syntax on line 16 appears nowhere else in the codebase and is likely a typo. Change "#{signerUrl}" to "${signerUrl}" for consistency.
🤖 Prompt for AI Agents
In `@config-overrides.example/genvm-module-web.yaml` around lines 13 - 17, The
signer_url template value uses the wrong interpolation syntax; locate the
signer_url entry (key: signer_url) where its value is currently "#{signerUrl}"
and change it to use the repository's standard "${signerUrl}" interpolation so
it matches other YAML configs.
| def make_snapshot(addr: str, slots_per_contract: int, value_size: int): | ||
| state = {} | ||
| for i in range(slots_per_contract): | ||
| slot = i.to_bytes(32, "big") | ||
| seed = i.to_bytes(4, "big") | ||
| value = (seed * ((value_size + 3) // 4))[:value_size] | ||
| state[b64(slot)] = b64(value) | ||
| return SimpleNamespace(contract_address=addr, states={"accepted": state}, balance=0) | ||
|
|
||
|
|
||
| def make_view(snapshot_cls, primary_snapshot, snapshot_factory, shared_cache): | ||
| sig = inspect.signature(snapshot_cls.__init__) | ||
| kwargs = {"readonly": True} | ||
| if "shared_decoded_value_cache" in sig.parameters: | ||
| kwargs["shared_decoded_value_cache"] = shared_cache | ||
| return snapshot_cls(primary_snapshot, snapshot_factory, **kwargs) | ||
|
|
||
|
|
||
| def build_slots(slots_per_contract: int): | ||
| return [i.to_bytes(32, "big") for i in range(slots_per_contract)] | ||
|
|
||
|
|
||
| def run_scenario( | ||
| scenario, | ||
| snapshot_cls, | ||
| address_cls, | ||
| executions: int, | ||
| use_shared_cache: bool, | ||
| ): | ||
| slots = build_slots(scenario["slots_per_contract"]) | ||
|
|
||
| primary_addr = ("0x" + "ab" * 20).lower() | ||
| primary_snapshot = make_snapshot( | ||
| primary_addr, scenario["slots_per_contract"], scenario["local_value_size"] | ||
| ) | ||
|
|
||
| cross_addrs = [] | ||
| cross_snapshots = {} | ||
| for i in range(scenario["cross_contracts"]): | ||
| addr = ("0x" + format(i + 1, "040x")).lower() | ||
| cross_addrs.append(addr) | ||
| cross_snapshots[addr] = make_snapshot( | ||
| addr, scenario["slots_per_contract"], scenario["cross_value_size"] | ||
| ) | ||
|
|
||
| snapshot_calls = {"count": 0} | ||
|
|
||
| def snapshot_factory(addr_hex: str): | ||
| snapshot_calls["count"] += 1 | ||
| addr_hex = addr_hex.lower() | ||
| if addr_hex == primary_addr: | ||
| return primary_snapshot | ||
| return cross_snapshots[addr_hex] | ||
|
|
||
| primary_address = address_cls(primary_addr) | ||
| cross_addresses = [address_cls(a) for a in cross_addrs] | ||
|
|
||
| shared_cache = {} if use_shared_cache else None | ||
| per_exec_ms = [] | ||
| agg_metrics = defaultdict(int) | ||
|
|
||
| local_reads = scenario["local_reads"] | ||
| cross_reads = scenario["cross_reads"] | ||
| total_reads = local_reads + cross_reads | ||
| local_stride = scenario["local_stride"] | ||
| cross_stride = scenario["cross_stride"] | ||
|
|
||
| for _ in range(executions): | ||
| view = make_view(snapshot_cls, primary_snapshot, snapshot_factory, shared_cache) | ||
| t0 = time.perf_counter() | ||
|
|
||
| local_done = 0 | ||
| cross_done = 0 | ||
| for i in range(total_reads): | ||
| do_cross = (cross_done < cross_reads) and ( | ||
| local_done >= local_reads or (i % 5 == 0) | ||
| ) | ||
| if do_cross: | ||
| cidx = cross_done % len(cross_addresses) if cross_addresses else 0 | ||
| sidx = (cross_done * cross_stride) % len(slots) | ||
| view.storage_read( | ||
| cross_addresses[cidx], slots[sidx], 0, scenario["cross_read_len"] | ||
| ) | ||
| cross_done += 1 | ||
| else: | ||
| sidx = (local_done * local_stride) % len(slots) | ||
| view.storage_read( | ||
| primary_address, slots[sidx], 0, scenario["local_read_len"] | ||
| ) | ||
| local_done += 1 | ||
|
|
||
| per_exec_ms.append((time.perf_counter() - t0) * 1000) | ||
|
|
||
| if hasattr(view, "get_metrics"): | ||
| metrics = view.get_metrics() | ||
| for k, v in metrics.items(): | ||
| if isinstance(v, int): | ||
| agg_metrics[k] += v | ||
|
|
||
| warm = per_exec_ms[1:] if len(per_exec_ms) > 1 else per_exec_ms | ||
| return { | ||
| "scenario": scenario["name"], | ||
| "use_shared_cache": use_shared_cache, | ||
| "executions": executions, | ||
| "first_exec_ms": round(per_exec_ms[0], 2), | ||
| "warm_mean_ms": round(mean(warm), 2), | ||
| "mean_exec_ms": round(mean(per_exec_ms), 2), | ||
| "min_exec_ms": round(min(per_exec_ms), 2), | ||
| "max_exec_ms": round(max(per_exec_ms), 2), | ||
| "total_ms": round(sum(per_exec_ms), 2), | ||
| "snapshot_factory_calls": snapshot_calls["count"], | ||
| "shared_cache_size": len(shared_cache) if shared_cache is not None else 0, | ||
| "metrics": dict(agg_metrics), | ||
| } | ||
|
|
||
|
|
||
| def main(): | ||
| parser = argparse.ArgumentParser() | ||
| parser.add_argument("--executions", type=int, default=7) | ||
| args = parser.parse_args() |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cat -n scripts/benchmarks/snapshot_bench.py | head -150Repository: genlayerlabs/genlayer-studio
Length of output: 6103
🏁 Script executed:
cat -n scripts/benchmarks/snapshot_bench.py | sed -n '38,45p'Repository: genlayerlabs/genlayer-studio
Length of output: 302
🏁 Script executed:
# Check if there are any other benchmark files to see how typing is typically done
fd "bench" -e py | head -10Repository: genlayerlabs/genlayer-studio
Length of output: 187
🏁 Script executed:
cat -n scripts/benchmarks/validator_batch_bench.py | head -100Repository: genlayerlabs/genlayer-studio
Length of output: 3785
🏁 Script executed:
# Check the rest of validator_batch_bench.py to see the main function
tail -50 scripts/benchmarks/validator_batch_bench.pyRepository: genlayerlabs/genlayer-studio
Length of output: 1720
🏁 Script executed:
# Check Python version targeting in the repository to validate typing syntax
grep -r "python_requires\|target-version" . --include="*.toml" --include="*.cfg" | head -5Repository: genlayerlabs/genlayer-studio
Length of output: 54
🏁 Script executed:
cat pyproject.toml | grep -A 5 -B 5 "python"Repository: genlayerlabs/genlayer-studio
Length of output: 118
🏁 Script executed:
find . -maxdepth 2 -name "*.toml" -o -name "*.cfg" -o -name "setup.py" | grep -v node_modules | head -10Repository: genlayerlabs/genlayer-studio
Length of output: 83
🏁 Script executed:
cat ruff.tomlRepository: genlayerlabs/genlayer-studio
Length of output: 978
🏁 Script executed:
# Check for Python version specification and typing patterns in existing code
find . -name "*.py" -path "*/backend/*" -type f | head -5 | xargs grep -l "from typing import\|Union\|Optional" | head -3Repository: genlayerlabs/genlayer-studio
Length of output: 133
🏁 Script executed:
head -30 ./backend/consensus/base.pyRepository: genlayerlabs/genlayer-studio
Length of output: 1019
🏁 Script executed:
# Check if dict[str, Any] syntax (PEP 585) is used elsewhere in the codebase to confirm Python 3.12 compatibility
grep -r "dict\[" backend/ --include="*.py" | head -3Repository: genlayerlabs/genlayer-studio
Length of output: 290
Add missing type hints to benchmark helper functions.
The file violates the coding guideline "Include type hints in all Python code" for **/*.py files. Functions make_snapshot(), make_view(), build_slots(), run_scenario() parameters scenario and snapshot_cls and address_cls, and main() lack required type annotations.
✍️ Suggested typing pass
import time
from collections import defaultdict
from statistics import mean
from types import SimpleNamespace
+from typing import Any, Callable
-def make_snapshot(addr: str, slots_per_contract: int, value_size: int):
+def make_snapshot(
+ addr: str, slots_per_contract: int, value_size: int
+) -> SimpleNamespace:
state = {}
for i in range(slots_per_contract):
slot = i.to_bytes(32, "big")
seed = i.to_bytes(4, "big")
value = (seed * ((value_size + 3) // 4))[:value_size]
state[b64(slot)] = b64(value)
return SimpleNamespace(contract_address=addr, states={"accepted": state}, balance=0)
-def make_view(snapshot_cls, primary_snapshot, snapshot_factory, shared_cache):
+def make_view(
+ snapshot_cls: type,
+ primary_snapshot: SimpleNamespace,
+ snapshot_factory: Callable[[str], SimpleNamespace],
+ shared_cache: dict[str, bytes] | None,
+) -> Any:
sig = inspect.signature(snapshot_cls.__init__)
kwargs = {"readonly": True}
if "shared_decoded_value_cache" in sig.parameters:
kwargs["shared_decoded_value_cache"] = shared_cache
return snapshot_cls(primary_snapshot, snapshot_factory, **kwargs)
-def build_slots(slots_per_contract: int):
+def build_slots(slots_per_contract: int) -> list[bytes]:
return [i.to_bytes(32, "big") for i in range(slots_per_contract)]
def run_scenario(
- scenario,
- snapshot_cls,
- address_cls,
+ scenario: dict[str, int | str],
+ snapshot_cls: type,
+ address_cls: type,
executions: int,
use_shared_cache: bool,
-) -> dict[str, Any]:
+) -> dict[str, Any]:
slots = build_slots(scenario["slots_per_contract"])
...
-def main():
+def main() -> None:
parser = argparse.ArgumentParser()🤖 Prompt for AI Agents
In `@scripts/benchmarks/snapshot_bench.py` around lines 16 - 135, Add explicit
type hints to the benchmark helpers: annotate make_snapshot(addr: str,
slots_per_contract: int, value_size: int) -> SimpleNamespace;
make_view(snapshot_cls: Type, primary_snapshot: SimpleNamespace,
snapshot_factory: Callable[[str], Any], shared_cache: Optional[Mapping[Any,
Any]] = None) -> Any (or the snapshot instance type);
build_slots(slots_per_contract: int) -> List[bytes]; run_scenario(scenario:
Dict[str, Any], snapshot_cls: Type, address_cls: Type, executions: int,
use_shared_cache: bool) -> Dict[str, Any]; and main() -> None; and add imports
for typing symbols used (Type, Callable, Any, Optional, Mapping, List, Dict) so
static checkers accept the annotations. Ensure default values/signatures remain
unchanged while adding the return types.
| local_done = 0 | ||
| cross_done = 0 | ||
| total_reads = local_reads + cross_reads | ||
| for i in range(total_reads): | ||
| do_cross = (cross_done < cross_reads) and ( | ||
| local_done >= local_reads or (i % 5 == 0) | ||
| ) | ||
| if do_cross: | ||
| cidx = cross_done % len(cross_addresses) | ||
| sidx = (cross_done * 13) % len(slots) | ||
| view.storage_read(cross_addresses[cidx], slots[sidx], 0, 512) | ||
| cross_done += 1 | ||
| else: | ||
| sidx = (local_done * 7) % len(slots) | ||
| view.storage_read(primary_address, slots[sidx], 0, 32) | ||
| local_done += 1 |
There was a problem hiding this comment.
ZeroDivisionError when cross_contracts=0 but cross_reads > 0, or slots_per_contract=0.
cross_addresses is empty if --cross-contracts 0, so len(cross_addresses) on line 105 causes a division-by-zero. Same for len(slots) on lines 106/110 if --slots 0.
Proposed fix — validate inputs
def run_mode(
mode,
snapshot_cls,
address_cls,
*,
executions,
slots_per_contract,
cross_contracts,
local_reads,
cross_reads,
factory_delay_ms,
):
+ if slots_per_contract <= 0:
+ raise ValueError("slots_per_contract must be > 0")
+ if cross_reads > 0 and cross_contracts <= 0:
+ raise ValueError("cross_contracts must be > 0 when cross_reads > 0")🤖 Prompt for AI Agents
In `@scripts/benchmarks/validator_batch_bench.py` around lines 97 - 112, The loop
can raise ZeroDivisionError when cross_addresses or slots are empty; validate
inputs before the loop and guard modulo/division uses: ensure that if
cross_reads > 0 then cross_addresses is not empty (or reduce/skip cross_reads),
and ensure slots (from slots_per_contract) is non-empty before using len(slots)
in sidx calculations; update the code around variables cross_addresses, slots,
cross_reads, local_reads, cross_done and the for-loop that computes sidx/cidx to
either validate/raise a clear error earlier or branch to skip cross reads and
local reads when their corresponding address/slot lists are empty so no modulo
by zero occurs.
| # { "Depends": "py-genlayer:1jb45aa8ynh2a9c9xn3b7qqh8sm5q93hwfp7jqmwsfhh8jpz09h6" } | ||
| from genlayer import * |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cat -n tests/direct/storage_read_bench.pyRepository: genlayerlabs/genlayer-studio
Length of output: 808
Replace star import with explicit imports for clarity and to resolve Ruff lint flags F403/F405.
The file uses gl, u256, and DynArray from genlayer. Star imports obscure dependencies and can cause linting errors; explicit imports make the code's requirements clear.
♻️ Proposed change
-from genlayer import *
+from genlayer import gl, u256, DynArray📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # { "Depends": "py-genlayer:1jb45aa8ynh2a9c9xn3b7qqh8sm5q93hwfp7jqmwsfhh8jpz09h6" } | |
| from genlayer import * | |
| # { "Depends": "py-genlayer:1jb45aa8ynh2a9c9xn3b7qqh8sm5q93hwfp7jqmwsfhh8jpz09h6" } | |
| from genlayer import gl, u256, DynArray |
🧰 Tools
🪛 Ruff (0.15.0)
[error] 2-2: from genlayer import * used; unable to detect undefined names
(F403)
🤖 Prompt for AI Agents
In `@tests/direct/storage_read_bench.py` around lines 1 - 2, Replace the star
import in this test module with explicit imports to remove F403/F405 lint flags
and clarify dependencies: import the specific symbols used (gl, u256, DynArray)
from genlayer (i.e., replace "from genlayer import *" with an explicit import
that lists gl, u256, DynArray) so the test only exposes the needed names.
|
🎉 This PR is included in version 0.99.1 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
Summary
GENVM_TAGfrom Dockerfile), saving ~2.5 min on warm runsdocker-compose.ci.ymloverlay to mount host cache into jsonrpc + consensus-worker containerssetup-pythonwithcache-dependency-pathfor all 3 requirements files (~30-45s savings)pip installcommands into 1 (single dependency resolution pass)-n 4to-n 8(tests are I/O-bound, waiting on RPC responses)tests/db-sqlalchemy/validators_registry_test.py(<1s, no GenVM overhead)Test plan
Summary by CodeRabbit