Repository navigation
Performance of referencing library vs deprecated jsonschema.RefResolver is very bad when there are a lot of references in schema #178
Description
Activity
Hey there, I'm happy to have a look at this at some point, but is there a reason you're benchmarking against such an old version? Lots has changed since 4.18, so it'd be good if you shared numbers which were on 4.23.
Sorry, I didn't mention that I tested on every version between 4.18 and 4.23 to see if any had better performance. None of the versions past 4.18 improve the performance noticeably.
On 4.23, the results are actually a bit worse:
For the single test: 55s
For the 26 tests: 6m33sWe have also experienced similar performance issue in one of our tool after switching from RefResolver to this library. This is the commit in our library: PolusAI/sophios#287
We may be encountering the same issue, according to the profiler. We are still investigating
Version:referencing==0.35.1
From the profiler:ncalls tottime percall cumtime percall filename:lineno(function) 8257 0.027 0.000 6.939 0.001 <path>/.venv/lib/python3.12/site-packages/referencing/_core.py:643(lookup) 8257 0.027 0.000 6.495 0.001 <path>/.venv/lib/python3.12/site-packages/referencing/_core.py:405(get_or_retrieve) 5187 0.923 0.000 6.087 0.001 <path>/.venv/lib/python3.12/site-packages/referencing/_core.py:485(crawl) 907475 0.340 0.000 3.505 0.000 <path>/.venv/lib/python3.12/site-packages/referencing/_core.py:501(<genexpr>) 907475 0.486 0.000 3.165 0.000 <path>/.venv/lib/python3.12/site-packages/referencing/_core.py:235(<genexpr>)There seems to have been significant improvement here. Not sure if it's in jsonschema or referencer, but using latest jsonschema and referencer libs, the slowdown went from the previously reported ~35x to only ~2x. By adding a cache, I was able to get it down to comprable performance.
PR where I did that here in case it helps anyone experiencing this issue: https://github.com/Benchling-Open-Source/allotropy/pull/1217/changes
Hello!
I have a library that formats scientific data into a JSON schema called the
Allotrope Standard Model(ASM)The validation schemas are fairly large and complicated compared to other schemas I've seen in discussion boards, and are very modular, meaning there are a lot of references. In
allotropywe store the ASM schemas directly, and remove all remote references, replacing them with local references under$defs.We are finding that validating against the schemas using
jsonschemaversion4.18.0takes ~20x longer than4.17.0.As a concrete example:
Validating this data: https://raw.githubusercontent.com/Benchling-Open-Source/allotropy/refs/heads/main/tests/parsers/moldev_softmax_pro/testdata/MD_SMP_luminescence_endpoint_example08.json
Against this schema: https://github.com/Benchling-Open-Source/allotropy/blob/main/src/allotropy/allotrope/schemas/adm/plate-reader/REC/2024/06/plate-reader.schema.json
takes
~3.5son4.17.0and~55son4.18.0This translates to a runtime for all 26 tests in
tests/parsers/moldev_softmax_proof~30sin4.17.0to~6min4.18.0