Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions crates/modules-interfaces/src/domain.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ pub struct ExecutionData {
pub message: super::abi_stub::MessageData,
pub host_data: String,
pub code: Option<Bytes>,
pub leader_nondet_results: Option<Vec<Bytes>>,
pub leader_public_data: Option<Bytes>,
/// Maps each host method (by index) to a host id. When empty, all methods use host 0.
pub method_hosts: Vec<u8>,
/// Raw bytes written to each host connection before the first method byte.
Expand Down Expand Up @@ -408,7 +408,7 @@ pub struct ReportedResult {
pub emissions: Vec<ExecutionEmission>,

pub nondet_disagreement: Option<u32>,
pub nondet_results: Vec<bytes::Bytes>,
pub leader_public_data: bytes::Bytes,

pub data_fees_remaining: Vec<primitive_types::U256>,
pub data_fees_consumed: BucketsConsumed,
Expand Down
2 changes: 1 addition & 1 deletion docs/adr/013. pre-validating untrusted decoded inputs.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ untrusted entry calldata is decoded:

The `is_sync`/`is_leader()` boolean pair is replaced by `rt::RunMode { Sync,
Leader, Validator }`, fixed at startup from the execution data (`sync` flag;
else presence of `leader_nondet_results`). Rule 1 governs `Validator`/`Sync`
else presence of `leader_public_data`). Rule 1 governs `Validator`/`Sync`
(in sync there is no vote — the derived result is simply the call's result);
Rule 1b governs `Leader`.

Expand Down
2 changes: 1 addition & 1 deletion docs/adr/014. manager host socket protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ result without round-tripping it through the node. For a top-level run, the
manager validates the outer framing and decoded `ReportedResult` before
retaining it. Malformed reports are refused; a fatal result triggers a debug
assertion and is logged and downgraded to `VMError` in release builds. Embedded
`nondet_results` remain opaque because their encoding belongs to the executor
`leader_public_data` remains opaque because its encoding belongs to the executor
line

### HTTP surface
Expand Down
2 changes: 1 addition & 1 deletion docs/adr/015. cross-major contract calls.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ computed by the caller and carried on the wire. The table is complete against
| `host_data`, `gas_data` | Copied by the manager from the outer request, not carried in the envelope. |
| `method_hosts` | Constructed by the manager: host 0 by default, `consume_result` and `run_nested` on host 1. |
| `host_hello_data` | Copied from the outer request; entry 1 is cleared for the new manager socketpair. |
| Effectful and nondeterministic allocations | Cleared: `leader_nondet_results` and `record_actions` are empty because a deterministic child produces neither; `bucket_totals`, `message_fee_allocation` and `initial_time_units_allocation` are empty or zero because the child imports its remaining deterministic budget instead of being granted a fresh one. |
| Effectful and nondeterministic allocations | Cleared: `leader_public_data` and `record_actions` are empty because a deterministic child produces neither; `bucket_totals`, `message_fee_allocation` and `initial_time_units_allocation` are empty or zero because the child imports its remaining deterministic budget instead of being granted a fresh one. |
| `remaining_recursion` | Parent budget minus one. Carried as a remainder, not a spent count, so the bound stays the one the chain's root minted even when the lines disagree on `VM_RECURSION` -- but the receiving executor clamps it to its own limit, since a remainder is not an authority to exceed one. The manager clamps it again, to `CROSS_MAJOR_RECURSION` (6), so a delegated chain is bounded far more tightly than an in-process one: a hop costs a whole executor process, and the manager is the only party that sees the chain as a whole. The clamp also bounds the subtree below a delegated call, which is intended -- a chain that has already crossed a boundary does not get to recurse deeply in-process either. Also set, for a chain root, by the `initial_recursion` debug override. |
| `nested.memory_limit` | The derived limiter's remaining memory, made an explicit process-spawn limit. |
| `nested.stack` | Caller stack with the caller contract appended. |
Expand Down
30 changes: 7 additions & 23 deletions docs/website/src/impl-spec/02-vm/03-consensus.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,15 @@ Roles and the ``is_leader`` flag
A GenVM run is either a *leader* run or a *validator* run, decided once when the
node starts the run. The supervisor exposes this as
``Supervisor::is_leader()`` (``executor/src/rt/supervisor/mod.rs:209``); the flag is
derived from whether the node has handed in a vector of leader non-deterministic
results:
derived from whether the node has handed in opaque leader public data:

- Leader run: ``leader_nondet_results == None``. Every ``RunNondet`` call executes the
- Leader run: ``leader_public_data == None``. Every ``RunNondet`` call executes the
leader sub-program and the result is pushed into the supervisor's
``nondet_results`` vector, indexed by ``call_no``.
- Validator run: ``leader_nondet_results == Some(vec)``. ``RunNondet`` retrieves the
leader's result at ``call_no`` from this vector and feeds it back to the contract
as the second argument of the non-det block. If the leader produced fewer entries
than the validator demands, the run aborts with
- Validator run: ``leader_public_data == Some(bytes)``. The executor decodes the
bytes into the leader's results. ``RunNondet`` retrieves the result at ``call_no``
and feeds it back to the contract as the second argument of the non-det block. If
the leader produced fewer entries than the validator demands, the run aborts with
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
``VmError::leader_fault().nondet_output().absent()``.

``call_no`` is a monotonically increasing counter incremented per ``RunNondet``
Expand All @@ -32,22 +31,7 @@ a leader's i-th non-det result to the validator's i-th non-det check — the
ordering of ``RunNondet`` calls in deterministic code MUST match between leader
and validator, otherwise the contract is non-replayable.

Leader Output Format
--------------------

Each entry in the leader's ``nondet_results`` vector is a single ``ResultCode``
byte (the ``public_abi::ResultCode`` enum: ``Return``, ``UserError``, ``VmError``)
followed by an encoded payload:

- ``Return`` — raw bytes returned by the non-det block.
- ``UserError`` — either a calldata-encoded value (when the first four bytes are
zero, indicating a calldata prefix) or a UTF-8 string fallback.
- ``VmError`` — UTF-8 error code from
:doc:`/spec/appendix/constants` ``vm_error``.

The validator reconstructs a catchable contract outcome from these bytes and
the contract observes the same value as the leader did. Any other result code,
including ``FatalVmError``, is treated as malformed leader input
The leader data encoding is defined by :ref:`gvm-def-leader-output-format`.

Validator Comparison
--------------------
Expand Down
14 changes: 6 additions & 8 deletions docs/website/src/impl-spec/appendix/manager-api.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -780,7 +780,7 @@ components:
- host
- calldata
- code
- leader_nondet_results
- leader_public_data
- initial_time_units_allocation
properties:
selector:
Expand Down Expand Up @@ -843,15 +843,13 @@ components:
description: |
Contract bytecode for a deployment, or null for a call. Required to be
present; only its value is nullable.
leader_nondet_results:
type: array
leader_public_data:
type: string
format: binary
nullable: true
items:
type: string
format: binary
description: |
The leader's non-deterministic results for a validator run, or null for
a leader run. Required to be present; only its value is nullable.
Opaque executor-owned leader public data for a validator run, or null
for a leader run. Required to be present; only its value is nullable.
initial_time_units_allocation:
type: integer
format: uint32
Expand Down
4 changes: 2 additions & 2 deletions docs/website/src/impl-spec/appendix/manager-socket.rst
Original file line number Diff line number Diff line change
Expand Up @@ -270,8 +270,8 @@ debug manager asserts, while a release manager logs the executor violation and
rewrites both result-code locations to ``VmError`` before publication. Clients
therefore never receive a top-level ``FatalVmError``

The manager does not decode entries of the reported ``nondet_results`` vector.
Those bytes remain opaque, executor-line-specific consensus proposals
The manager does not decode reported ``leader_public_data``. The bytes remain
opaque, executor-line-specific consensus proposals

Lifecycle guarantees:

Expand Down
11 changes: 10 additions & 1 deletion docs/website/src/spec/03-vm/05-result.rst
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,16 @@ Non-Deterministic Block Result Encoding
These three are the only codes a leader-proposed non-deterministic block result
may carry; validators treat every other byte as a malformed leader result. A
fatal VM error computed by a leader's non-deterministic child propagates to its
caller and MUST NOT be encoded into ``nondet_results``
caller and MUST NOT be included in ``LeaderPublicData``

.. _gvm-def-leader-output-format:

Leader Output Format
--------------------

``LeaderPublicData`` is encoded as an RLP list containing every non-deterministic
block result, in execution order, followed by the byte string ``padded``. Each
result uses :ref:`gvm-def-subvm-result-encoding`

Contract Result Encoding
------------------------
Expand Down
16 changes: 8 additions & 8 deletions implementation/src/manager/run.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1003,7 +1003,7 @@ pub struct Request {
#[serde(default)]
#[calldata(default = default_unsafe_overrides)]
pub unsafe_overrides: UnsafeOverrides,
pub leader_nondet_results: Option<Vec<bytes::Bytes>>,
pub leader_public_data: Option<bytes::Bytes>,
/// Host-provided `node` fee constants (moved off `host_data`).
#[serde(default)]
#[calldata(default = default_gas_data)]
Expand Down Expand Up @@ -1512,7 +1512,7 @@ fn nested_effect(reported: &genvm_modules_interfaces::ReportedResult) -> Option<
storage_deltas,
emissions,
nondet_disagreement,
nondet_results,
leader_public_data,
data_fees_consumed:
genvm_modules_interfaces::BucketsConsumed {
storage,
Expand All @@ -1533,8 +1533,8 @@ fn nested_effect(reported: &genvm_modules_interfaces::ReportedResult) -> Option<
if nondet_disagreement.is_some() {
return Some("nondet_disagreement");
}
if !nondet_results.is_empty() {
return Some("nondet_results");
if !leader_public_data.is_empty() {
return Some("leader_public_data");
}
if !llm_consumed_gen_wei.is_zero() {
return Some("llm_consumed_gen_wei");
Expand Down Expand Up @@ -1944,7 +1944,7 @@ fn execution_data_from_request(req: &Request) -> genvm_modules_interfaces::Execu
message: req.message.clone(),
host_data: req.host_data.clone(),
code: req.code.clone(),
leader_nondet_results: req.leader_nondet_results.clone(),
leader_public_data: req.leader_public_data.clone(),
host_hello_data: req.host_hello_data.clone(),
method_hosts,
bucket_totals: req.bucket_totals.clone(),
Expand Down Expand Up @@ -2165,7 +2165,7 @@ impl Ctx {
permissions,
no_modules: true,
unsafe_overrides: Default::default(),
leader_nondet_results: None,
leader_public_data: None,
gas_data: parent_req.gas_data.clone(),
message_fee_allocation: Vec::new(),
initial_time_units_allocation: 0,
Expand All @@ -2179,7 +2179,7 @@ impl Ctx {
};
let mut execution_data = execution_data_from_request(&req);
execution_data.code = None;
execution_data.leader_nondet_results = None;
execution_data.leader_public_data = None;
execution_data.record_actions.clear();
execution_data.message_fee_allocation.clear();
execution_data.remaining_recursion =
Expand Down Expand Up @@ -2366,7 +2366,7 @@ async fn run_genvm_process(

let execution_context = if req.needs_modules() {
let host_data: genvm_modules_interfaces::HostData = serde_json::from_str(&req.host_data)?;
let role = if req.leader_nondet_results.is_none() {
let role = if req.leader_public_data.is_none() {
genvm_modules_interfaces::Role::Leader
} else {
genvm_modules_interfaces::Role::Validator
Expand Down
10 changes: 5 additions & 5 deletions implementation/src/manager/run_test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -553,7 +553,7 @@ fn clean_reported() -> genvm_modules_interfaces::ReportedResult {
storage_deltas: Vec::new(),
emissions: Vec::new(),
nondet_disagreement: None,
nondet_results: Vec::new(),
leader_public_data: bytes::Bytes::new(),
data_fees_remaining: Vec::new(),
data_fees_consumed: Default::default(),
llm_consumed_gen_wei: primitive_types::U256::zero(),
Expand Down Expand Up @@ -665,8 +665,8 @@ fn nested_reply_refuses_every_reported_effect() {
}),
("emissions", |r| r.emissions.push(some_emission())),
("nondet_disagreement", |r| r.nondet_disagreement = Some(0)),
("nondet_results", |r| {
r.nondet_results.push(bytes::Bytes::from_static(&[1]))
("leader_public_data", |r| {
r.leader_public_data = bytes::Bytes::from_static(&[1])
}),
("llm_consumed_gen_wei", |r| {
r.llm_consumed_gen_wei = primitive_types::U256::one()
Expand Down Expand Up @@ -802,7 +802,7 @@ fn request_field_names() -> Vec<&'static str> {
permissions,
no_modules,
unsafe_overrides,
leader_nondet_results,
leader_public_data,
gas_data,
message_fee_allocation,
initial_time_units_allocation,
Expand Down Expand Up @@ -830,7 +830,7 @@ fn request_field_names() -> Vec<&'static str> {
"permissions",
"no_modules",
"unsafe_overrides",
"leader_nondet_results",
"leader_public_data",
"gas_data",
"message_fee_allocation",
"initial_time_units_allocation",
Expand Down
28 changes: 15 additions & 13 deletions tests/runner/genvm_tool_plugins/integration.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
from gvm_extra.mock_host import MockHost as MockHost
from gvm_extra.mock_host import MockStorage as MockStorage
from origin.calldata import Address
from origin.leader_public_data import LeaderPublicData

if typing.TYPE_CHECKING:
import _jsonnet
Expand Down Expand Up @@ -74,6 +75,7 @@ def _is_ignore_hash_enabled() -> bool:
# version-independent and lives in the manager root.
TEMPLATES_DIR = local_ctx.shared.root_dir.joinpath('tests', 'templates')


# Default environment for tests
default_env = {
k: v
Expand Down Expand Up @@ -636,7 +638,7 @@ async def _run_single_step(
)

path_to_which_leader_puts_result = my_tmp_dir.parent.joinpath(
my_tmp_dir.name[:-1] + '_leader_nondet.pickle'
my_tmp_dir.name[:-1] + '_leader_public_data.bin'
)

# Set up paths
Expand All @@ -646,18 +648,18 @@ async def _run_single_step(

is_leader = single_conf.get('mode') == 'l'

# Load leader nondet results if available (for v/s modes)
# Load leader public data if available (for v/s modes)
if is_leader:
leader_nondet = None
leader_public_data = None
else:
leader_nondet = single_conf.get('leader_nondet', None)
if leader_nondet is None:
leader_nondet = pickle.loads(
await gvm_io.read_file_bytes(path_to_which_leader_puts_result)
leader_nondet_fixture = single_conf.get('leader_nondet', None)
if leader_nondet_fixture is None:
leader_public_data = await gvm_io.read_file_bytes(
path_to_which_leader_puts_result
)
if leader_nondet is not None:
else:
encoded_nondet = []
for res in leader_nondet:
for res in leader_nondet_fixture:
if isinstance(res, (bytes, bytearray, memoryview)):
encoded_nondet.append(bytes(res))
elif res['kind'] == 'return':
Expand Down Expand Up @@ -688,7 +690,7 @@ async def _run_single_step(
encoded_nondet.append(bytes(res['value']))
else:
raise ValueError(f'unknown leader_nondet kind: {res["kind"]}')
leader_nondet = encoded_nondet
leader_public_data = LeaderPublicData(encoded_nondet).encode()

# Create mock host
running_address = single_conf['message']['contract_address']
Expand Down Expand Up @@ -785,7 +787,7 @@ def resolve_executor(address, state, advisory_major):
calldata=calldata_bytes,
bucket_totals=bucket_totals,
gas_data=single_conf.get('gas_data'),
leader_nondet_results=leader_nondet,
leader_public_data=leader_public_data,
message_fee_allocation=message_fee_allocation,
unsafe_overrides=base_host.UnsafeOverrides(
reroute_to=single_conf.get('reroute_to', reroute_to)
Expand Down Expand Up @@ -852,10 +854,10 @@ def resolve_executor(address, state, advisory_major):

# Save leader nondet results for v/s modes
if is_leader:
nondet_list = res.result_nondet_results
leader_public_data = res.result_leader_public_data

await gvm_io.write_file_bytes(
path_to_which_leader_puts_result, pickle.dumps(nondet_list)
path_to_which_leader_puts_result, leader_public_data
)

# Save outputs
Expand Down
12 changes: 6 additions & 6 deletions tests/runner/origin/base_host.py
Original file line number Diff line number Diff line change
Expand Up @@ -502,7 +502,7 @@ class ConsumedResult:
result_fingerprint: ResultFingerprint | None = None
result_storage_deltas: list[tuple[bytes, bytes]] = field(default_factory=list)
result_emissions: list[ResultEmission] = field(default_factory=list)
result_nondet_results: list[bytes] = field(default_factory=list)
result_leader_public_data: bytes = b''
data_fees_remaining: list[int] = field(default_factory=list)

@classmethod
Expand Down Expand Up @@ -551,7 +551,7 @@ def decode(cls, raw: typing.Any) -> 'ConsumedResult':
result_fingerprint=decoded.get('fingerprint'),
result_storage_deltas=decoded.get('storage_deltas', []),
result_emissions=decoded.get('emissions', []),
result_nondet_results=decoded.get('nondet_results', []),
result_leader_public_data=decoded.get('leader_public_data', b''),
data_fees_remaining=decoded.get('data_fees_remaining', []),
)

Expand All @@ -571,7 +571,7 @@ class RunHostAndProgramRes:
result_fingerprint: ResultFingerprint | None
result_storage_deltas: list[tuple[bytes, bytes]]
result_emissions: list[ResultEmission]
result_nondet_results: list[bytes]
result_leader_public_data: bytes
data_fees_remaining: list[int]
metrics: dict[str, typing.Any] | None = None
vm_error_description: str | None = None
Expand Down Expand Up @@ -1165,7 +1165,7 @@ async def run_genvm(
bucket_totals: list[int],
code: bytes | None = None,
calldata: bytes,
leader_nondet_results: list[bytes] | None = None,
leader_public_data: bytes | None = None,
message_fee_allocation: collections.abc.Sequence[fees.MessageAllocationNode] = (),
unsafe_overrides: UnsafeOverrides | None = None,
request_extra: collections.abc.Mapping[
Expand Down Expand Up @@ -1214,7 +1214,7 @@ async def run_genvm(
'extra_args': list(extra_args),
'code': code,
'calldata': calldata,
'leader_nondet_results': leader_nondet_results,
'leader_public_data': leader_public_data,
'bucket_totals': bucket_totals,
'gas_data': effective_gas_data,
'message_fee_allocation': list(message_fee_allocation),
Expand Down Expand Up @@ -1383,7 +1383,7 @@ async def cancel_on_shutdown():
result_fingerprint=consumed.result_fingerprint,
result_storage_deltas=consumed.result_storage_deltas,
result_emissions=consumed.result_emissions,
result_nondet_results=consumed.result_nondet_results,
result_leader_public_data=consumed.result_leader_public_data,
data_fees_remaining=consumed.data_fees_remaining,
vm_error_description=vm_error_description,
execution_time=time.time() - started_at,
Expand Down
Loading
Loading