Skip to content

Optimize DAG-CBOR encode/decode and CAR block parsing#104

Merged
MarshalX merged 2 commits into
mainfrom
optimize-encode-decode
May 25, 2026
Merged

Optimize DAG-CBOR encode/decode and CAR block parsing#104
MarshalX merged 2 commits into
mainfrom
optimize-encode-decode

Conversation

@MarshalX

Copy link
Copy Markdown
Owner
  • Drop BufWriter<Vec<u8>>. Write directly into Vec<u8> (VecWriter)
  • Pre-filter CID candidates by leading byte before Cid::try_from (looks_like_cid)
  • Single dict walk in map encode. No more keys() + values() + by-index lookup
  • Exact-type pointer dispatch (Py_TYPE == &PyXxx_Type) instead of is_instance_of MRO walks
  • _PyDict_NewPresized(len) for map decode (CPython-only via cfg(CPython) + build.rs; PyPy falls back to PyDict_New); skipped when len ≤ 8
  • Drop Cursor wrapper for Cid::read_bytes in decode_car, use &[u8] directly

@codspeed-hq

codspeed-hq Bot commented May 25, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 17.98%

⚡ 31 improved benchmarks
❌ 1 regressed benchmark
✅ 160 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation test_dag_cbor_decode_real_data[twitter.json] 6.5 ms 8 ms -18.41%
Simulation test_dag_cbor_encode[roundtrip01.json] 14 µs 12.2 µs +14.81%
Simulation test_dag_cbor_encode[roundtrip02.json] 16 µs 14 µs +13.98%
Simulation test_dag_cbor_encode[roundtrip03.json] 14.2 µs 12.1 µs +16.61%
Simulation test_dag_cbor_encode[roundtrip04.json] 15 µs 13.1 µs +14.46%
Simulation test_dag_cbor_encode[roundtrip05.json] 14.9 µs 12.2 µs +22.2%
Simulation test_dag_cbor_encode[roundtrip06.json] 13.2 µs 11.3 µs +17.36%
Simulation test_dag_cbor_encode[roundtrip07.json] 15 µs 12 µs +24.71%
Simulation test_dag_cbor_encode[roundtrip08.json] 15.1 µs 13.2 µs +14.57%
Simulation test_dag_cbor_encode[roundtrip09.json] 19.1 µs 13.5 µs +41.59%
Simulation test_dag_cbor_encode[roundtrip10.json] 19.3 µs 13.8 µs +39.69%
Simulation test_dag_cbor_encode[roundtrip11.json] 14.9 µs 13.1 µs +13.54%
Simulation test_dag_cbor_encode[roundtrip12.json] 15.2 µs 13.4 µs +13.95%
Simulation test_dag_cbor_encode[roundtrip13.json] 15.1 µs 13.6 µs +11.21%
Simulation test_dag_cbor_encode[roundtrip14.json] 15.7 µs 14 µs +12.2%
Simulation test_dag_cbor_encode[roundtrip15.json] 14.8 µs 13.1 µs +13.38%
Simulation test_dag_cbor_encode[roundtrip16.json] 18 µs 16.3 µs +10.33%
Simulation test_dag_cbor_encode[roundtrip17.json] 15.7 µs 13.9 µs +13.22%
Simulation test_dag_cbor_encode[roundtrip18.json] 15.1 µs 13.5 µs +12.4%
Simulation test_dag_cbor_encode[roundtrip19.json] 15.1 µs 13.4 µs +12.64%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing optimize-encode-decode (341f165) with main (2163f07)

Open in CodSpeed

@MarshalX

MarshalX commented May 25, 2026

Copy link
Copy Markdown
Owner Author
image image

Twitter decode is actually faster locally 6.4%. Probably the limitation of codspeed simulation

@MarshalX MarshalX merged commit 4c7905a into main May 25, 2026
4 of 5 checks passed
@MarshalX MarshalX deleted the optimize-encode-decode branch May 25, 2026 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant