Skip to content

perf(assets-controller): hand-rolled Multicall3 aggregate3 codec (~44× faster on Hermes) - #11010

Closed
Prithpal-Sooriya wants to merge 5 commits into
mainfrom
perf/multicall-aggregate3-hot-path
Closed

Prithpal-Sooriya wants to merge 5 commits into
mainfrom
perf/multicall-aggregate3-hot-path

Conversation

@Prithpal-Sooriya

@Prithpal-Sooriya Prithpal-Sooriya commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

MulticallClient uses the generic Ethers AbiCoder (Interface.encodeFunctionData / decodeFunctionResult) for every aggregate3 batch. On Hermes (React Native / Android) this dominates balance-fetch time: a 300-call batch costs ~25ms to encode, ~19ms to decode the response, and ~5.5ms to decode the per-result uint256 balances — measured on the Hermes engine directly.

This PR replaces that path with hand-rolled fixed-layout codecs:

  • encodeAggregate3 — direct string assembly with the aggregate3 selector hardcoded (and asserted against the Interface sighash at module load, so an ABI change fails loudly instead of silently producing stale calldata)
  • decodeAggregate3Response — substring parsing that follows every offset word (dynamic (bool, bytes) tuples require one cumulative offset word per result)
  • decodeUint256 — native BigInt(data.slice(0, 66)).toString()

Exported signatures are unchanged. encodeBalanceOf / encodeGetEthBalance still use Ethers deliberately: they're cached per account and off the per-batch hot path.

Benchmarks (Hermes engine, -O, median of 3 runs)

Stage (300-call batch) Before After
encodeAggregate3 25.0 ms 0.65 ms 38×
decodeAggregate3Response 18.4 ms 0.25 ms 74×
decodeUint256 5.5 ms 0.2 ms 27×
Full batch path 49.7 ms 1.125 ms ~44×

Measured with the bundled single-file Hermes bench in .auto/ (see below).

Not over-fit to the bench shape (scaling sweep)

The codec's per-call cost is flat across batch sizes and nearly independent of returnData size, so the ~44× is a property of the approach, not of the n=300 bench:

Batch size (n) Hand-rolled full path Per call
30 0.10 ms ~3.3 µs
300 1.125 ms ~3.8 µs
1500 5.69 ms ~3.8 µs
3000 11.17 ms ~3.7 µs
returnData per result (n=300) Hand-rolled decode viem decode
32 B (bench default) 0.275 ms 7.35 ms
500 B 0.325 ms (+18% for 15.6× the bytes) 20 ms (~linear in bytes)

One caveat in the baseline's favor: the bench uses all-lowercase addresses, so Ethers/viem skip EIP-55 checksum keccak validation — with real mixed-case addresses the library baselines get slower, i.e. ~44× is a conservative lower bound.

Experimental codec selection API (for on-device A/B)

This PR also adds an experimental seam so the mobile app can A/B all three codecs on a real device:

  • setAggregate3CodecVariant('handrolled' | 'viem' | 'ethers'), getAggregate3Codec(variant), getActiveAggregate3Codec() — exported from @metamask/assets-controller
  • Optional aggregate3Codec option on MulticallClient for explicit per-client injection
  • viem (^2.56.8, already in the monorepo) is the alternative codec; stock Ethers is the pre-PR behavior
  • The default remains the hand-rolled codec; the seam will be removed once the default is settled

Suggested on-device measurement (temporary, resolves before merge): call setAggregate3CodecVariant('viem' | 'ethers') (or inject aggregate3Codec: getAggregate3Codec(...)) before controller construction, then time the full balance-fetch wall-clock for a large token list per variant, and verify native + ERC-20 balances are unchanged for the fetched accounts (including unusual returnData lengths). For reference on the Hermes engine (-O, median of 3): stock Ethers ~49.7 ms → hand-rolled ~1.125 ms; viem ~11.85 ms for a 300-call batch.

Correctness

  • Encode output is byte-identical to Ethers — enforced by the existing jest golden test and by runtime parity self-checks on every bench run (including 1/31/100-byte callData edge lengths and mixed-case inputs)
  • Decode output identical to Ethers for all bench inputs (empty/1/31/100-byte returnData), and the result-count-mismatch error message is preserved verbatim
  • 0X-prefixed data is rejected — matching Ethers v5's arrayify behavior
  • New tests cover the added validation error paths (file coverage 98.8%); remaining uncovered lines are a load-time canary and a defensive guard

Verification

  • Full @metamask/assets-controller jest suite passes
  • Root yarn build clean
  • oxlint / oxfmt clean
  • auto-changelog validate clean
  • Mobile validation on a real Android device pending — the codec-selection API above makes this an on-device A/B between handrolled / viem / ethers; numbers here come from the Hermes engine directly

.auto/ playground (will be excluded before merge)

.auto/ contains the autoresearch experiment loop used to produce this change: a Hermes-runnable bench that extracts the real MulticallClient.ts (so measurements always reflect actual product code), bundles the ethers-v5 + controller-utils closure, and enforces parity self-checks vs stock Ethers; plus a keep/discard experiment runner (run-exp.sh), a dashboard-compatible verdict log (log.jsonl), and the idea ledger (ideas.md). Kept in the PR for now for transparency; it will be dropped before merge.


Note

Medium Risk
Changes how on-chain multicall calldata and RPC responses are built/parsed on a balance-fetch hot path; correctness relies on ABI layout parity with Ethers, though golden tests and new validation cases mitigate regressions.

Overview
Replaces generic Ethers AbiCoder usage on the Multicall3 aggregate3 hot path in MulticallClient with hand-rolled, fixed-layout encode/decode: encodeAggregate3 assembles calldata via string operations (hardcoded selector + load-time sighash guard), decodeAggregate3Response walks offsets with substring parsing, and decodeUint256 uses native BigInt. Inputs are validated up front with explicit errors; outputs stay byte-identical to Ethers for valid data.

Tests add coverage for invalid addresses/callData, malformed/truncated responses, and mixed-case returnData. CHANGELOG records the Hermes-oriented speedup (~44× on the bundled bench for a 300-call batch).

Also adds a .auto/ Hermes benchmark harness (extracts live MulticallClient code, bundles ethers closure, parity self-checks, measure.sh / run-exp.sh, experiment log)—intended as dev tooling; PR notes it may be dropped before merge.

Reviewed by Cursor Bugbot for commit b26d4cb. Bugbot is set up for automated code reviews on this repo. Configure here.

Replace the generic Ethers AbiCoder encoding/decoding used by
MulticallClient with fixed-layout codecs for the aggregate3 hot path:

- encodeAggregate3: direct string assembly (selector asserted against
  the interface sighash at load time)
- decodeAggregate3Response: substring parsing following all offset words
- decodeUint256: native BigInt

Output is byte-identical to Ethers (golden jest test + runtime parity
self-checks); error behavior for malformed data and result-count
mismatches is preserved. 300-call batch encode+decode+uint256:
49.7ms -> 1.125ms (~44x) on Hermes with -O, median of 3 runs.

Also adds error-path tests (file coverage 98.8%).
Experiment loop infrastructure used to produce this change:
- .auto/bench: extracts the real MulticallClient.ts, bundles an
  ethers-v5 + controller-utils closure into a single CJS file runnable
  by the Hermes binary, runs encode/decode/uint256 stages with parity
  self-checks vs stock Ethers
- .auto/run-exp.sh + log.jsonl: keep/discard verdicts (2% threshold),
  dashboard-compatible log
- .auto/ideas.md: idea ledger and learnings

NOTE: benchmark scaffolding only — will be excluded from this PR
before merge.
@Prithpal-Sooriya
Prithpal-Sooriya requested review from a team as code owners October 22, 2026 15:10
…m/ethers A/B)

Add an aggregate3Codecs module exposing setAggregate3CodecVariant /
getAggregate3Codec / getActiveAggregate3Codec and an optional
aggregate3Codec option on MulticallClient, so consumers (mobile) can
A/B the hand-rolled default against viem and stock Ethers on real
devices. Includes viem as a dependency, an it.each parity suite across
all three codecs, and the .auto scaling-sweep results showing the
hand-rolled codec is not over-fit to the n=300 bench shape. Also fixes
all oxlint violations in product files and excludes the temporary
.auto playground from oxlint.
Remeasured on the Hermes V1 engine React Native 0.86.3 ships
(hermes-v250829098.0.17) instead of the earlier Static Hermes checkout.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Prithpal-Sooriya

Copy link
Copy Markdown
Contributor Author

This has been polished and is being merged via #11078

Naz-Ovh pushed a commit to 0x-fork/metamask-core that referenced this pull request Nov 4, 2026
…11078)

## Explanation

Multicall3 `aggregate3` balance batches currently go through the generic
Ethers ABI coder. This adds fixed-layout encode and decode functions for
that calldata shape and uses them on the balance hot path, following the
approach in MetaMask#11010.

The previous Ethers functions remain exported as
`encodeAggregate3WithEthers`, `decodeAggregate3ResponseWithEthers`, and
`decodeUint256WithEthers`. Table-driven cases and 100 deterministic
pseudo-random batches check that the new codecs match those Ethers
results byte for byte, including empty, partial-word, and multiword call
data.

https://www.loom.com/share/c696e7903eba4e46ae75e2b8984b0230

## References

* Related to MetaMask#11010
* https://consensyssoftware.atlassian.net/browse/ASSETS-4166

## Checklist

- [x] I've updated the test suite for new or updated code as appropriate
- [x] I've updated documentation (JSDoc, Markdown, etc.) for new or
updated code as appropriate
- [x] I've communicated my changes to consumers by [updating changelogs
for packages I've
changed](https://github.com/MetaMask/core/tree/main/docs/processes/updating-changelogs.md)
- [ ] I've introduced [breaking
changes](https://github.com/MetaMask/core/tree/main/docs/processes/breaking-changes.md)
in this PR and have prepared draft pull requests for clients and
consumer packages to resolve them

## Test plan

- [x] `yarn workspace @metamask/assets-controller exec vitest run
--coverage.enabled=false
src/data-sources/evm-rpc-services/clients/MulticallClient.test.ts` (56
tests)
- [x] `yarn oxlint` on the changed MulticallClient files
- [x] `yarn workspace @metamask/assets-controller run
changelog:validate`
- [x] Confirm the hand-rolled and Ethers codecs stay byte-identical for
the pseudo-random cases in CI



<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Changes how on-chain balance batch calldata is built and parsed on
every RPC refresh; parity tests reduce but do not eliminate risk from a
hand-maintained ABI layout diverging from Ethers.
> 
> **Overview**
> **Multicall balance batches** no longer encode/decode Multicall3
`aggregate3` through the generic Ethers ABI coder on the hot path. A new
**`aggregate3Codec`** module writes and reads the fixed `aggregate3`
layout directly (`encodeAggregate3`, `decodeAggregate3Response`,
`decodeUint256`), and **`MulticallClient`** now calls those helpers when
building and parsing balance multicalls.
> 
> The previous Ethers implementations remain as
**`encodeAggregate3WithEthers`**,
**`decodeAggregate3ResponseWithEthers`**, and
**`decodeUint256WithEthers`** for reference and tests.
**`aggregate3Codec.test.ts`** asserts byte-for-byte parity with Ethers
(including 100 random batches and 300-call encode/decode cases). Package
exports for **`encodeAggregate3`** / **`decodeAggregate3Response`** now
come from the codec module. An architecture decision doc and changelog
entry document motivation (mobile encode/decode cost) and maintenance
tradeoffs.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
08db40a. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-assets area-performance Issues relating to slowness of app, cpu usage, and/or blank screens. assets-performance-agent Opened by the assets performance autoresearch agent DO-NOT-MERGE

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant