Skip to content

Tighten lightfuzz serial Error Resolution - #3427

Merged
liquidsec merged 2 commits into
devfrom
lightfuzz-serial-error-resolution
Oct 5, 2026
Merged

liquidsec merged 2 commits into
devfrom
lightfuzz-serial-error-resolution

Conversation

@liquidsec

Copy link
Copy Markdown
Contributor

The serial submodule's "Error Resolution" check works like this: put garbage in a parameter, get an error back, then send a real serialized object instead. If the error goes away, maybe the app deserialized it.

For that to mean anything, three things have to be true:

  1. the error was the app failing to parse what we sent
  2. the error is still there while we're testing
  3. the error went away because of what the payload is, not what it looks like

Only the third one was actually checked. This handles the other two.

Changes

Only treat a 500 as an error worth resolving. Any baseline status used to count. A 302 isn't an error, it's the app redirecting you. A 404 means the route didn't match. A 502 or 503 comes from a proxy or CDN having a bad moment, not from the app parsing anything. None of those can be "resolved" by deserializing something. This also replaces two one-off status checks with a single list.

Send all three requests together, twice. We used to re-send the control once to check the error was still there, then send the corrupted twin once, both after the fact. If a server flips between working and failing over a few seconds (rate limiting, a flaky CDN, an origin restarting), sending the same request again tells you nothing, because you're still sampling the same bad moment. Now the payload, the twin and the control all go out together, twice, and it has to come out the same way both times. Also, a request that never comes back used to count as "yep, still broken"; now it throws the candidate away.

Track the one-language rule per host instead of per parameter. A real sink speaks one format. A host that looks like it deserializes Java and PHP and Python isn't deserializing anything. We already threw those out, but only when they turned up on the same parameter, so a host that spread them across several parameters kept all of them. It's now tracked for the whole host, plus a cap on how many parameters on one host can report this at all.

Dropped an error string that was too generic. One of the strings in SERIALIZATION_ERRORS shows up on any login form that rejects a bad username, so it couldn't tell a broken deserializer from a wrong password.

A bug found along the way

The baseline status checks were continue on the payload loop, so they skipped the whole rest of the loop body, Differential Error Analysis included. Their comments only mentioned Error Resolution, but a real Java error string was unreportable whenever the baseline came back with the wrong status. Differential Error Analysis doesn't care what the baseline status is, it just needs a framework error string in the response that isn't in the baseline, so it now runs either way.

What this costs us

We'll miss a sink whose error shows up as a 502 through a proxy, and we'll miss one that behaves differently every time you hit it. Fair trade for a LOW confidence check. The OOB interactsh payloads are the ones that actually prove deserialization, they emit CONFIRMED, and nothing here touches them.

Tests

125 pass on test_module_lightfuzz.py. Six new tests, one per point above. I checked that each one fails if you revert the source and keep the tests, so they're testing the fix rather than just passing.

Two existing tests needed changing:

  • The multi-language test wasn't testing anything. Its fake server returned 200 for the control payload too, so the baseline was already 200, nothing ever looked suspicious, and the test asserted "no findings" against a scan that couldn't produce any. Its server now recognizes serialization magic headers, so the real payloads succeed and the controls and twins don't.
  • The differential-analysis false positive test was built around the error string I removed. It would still pass, but only because none of the remaining strings appear in it. I swapped it for one the submodule still looks for, present in both the baseline and the probe, so it goes on testing the "don't fire if the string is always there" guard. Its assertion is unchanged.

Error Resolution infers a deserializer from an error that a serialized
payload makes disappear. That inference needs the baseline error to be an
application-level failure to interpret the payload, and needs the error to
hold for as long as the test runs. Neither was checked.

- Gate the technique on an allowlist of baseline statuses. A redirect is not
  an error, 404 is routing, 401/403/429 are policy, and 5xx above 500 is
  upstream infrastructure.
- Replace the single control re-send and the separate scrambled-header twin
  probe with one trial that samples payload, twin and control together over
  several rounds. Server state that varies over seconds makes consecutive
  samples of one request correlated, so the arms have to be interleaved to be
  comparable. An unanswered sample is now inconclusive rather than
  confirmation.
- Track the single-language-family invariant per host rather than per
  parameter, and cap how many parameters per host may report the technique.
- Drop an error string too generic to tell a deserialization failure from a
  rejected login.

The baseline status guards previously sat on the payload loop, so they also
disabled Differential Error Analysis. That technique now runs whatever status
the baseline carries.
@codecov

codecov Bot commented Sep 7, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.12403% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 91%. Comparing base (1471146) to head (945b05e).
⚠️ Report is 6 commits behind head on dev.

Files with missing lines Patch % Lines
bbot/modules/lightfuzz/submodules/serial.py 89% 4 Missing ⚠️
.../test_step_2/module_tests/test_module_lightfuzz.py 99% 1 Missing ⚠️
Additional details and impacted files
@@          Coverage Diff          @@
##             dev   #3427   +/-   ##
=====================================
+ Coverage     91%     91%   +1%     
=====================================
  Files        460     460           
  Lines      48998   49094   +96     
=====================================
+ Hits       44132   44227   +95     
- Misses      4866    4867    +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

github-actions Bot commented Sep 7, 2026 •

Copy link
Copy Markdown
Contributor

📊 Performance Benchmark Report

Comparing dev (baseline) vs lightfuzz-serial-error-resolution (current)

📈 Detailed Results (All Benchmarks)

📋 Complete results for all benchmarks - includes both significant and insignificant changes

🧪 Test Name 📏 Base 📏 Current 📈 Change 🎯 Status
Bloom Filter Dns Mutation Tracking Performance 3.74ms 3.75ms +0.3% ⚪ ✅
Bloom Filter Large Scale Dns Brute Force 15.36ms 15.12ms -1.6% ⚪ ✅
Large Closest Match Lookup 360.98ms 371.29ms +2.9% ⚪ ✅
Realistic Closest Match Workload 199.70ms 196.39ms -1.7% ⚪ ✅
Event Memory Medium Scan 1402 B/event 1408 B/event +0.4% ⚪ ✅
Event Memory Large Scan 1527 B/event 1527 B/event +0.0% ⚪ ✅
Event Validation Full Scan Startup Small Batch 428.63ms 423.25ms -1.3% ⚪ ✅
Event Validation Full Scan Startup Large Batch 566.47ms 563.05ms -0.6% ⚪ ✅
Make Event Autodetection Small 26.90ms 26.97ms +0.3% ⚪ ✅
Make Event Autodetection Large 275.92ms 272.57ms -1.2% ⚪ ✅
Make Event Explicit Types 11.88ms 11.91ms +0.3% ⚪ ✅
Excavate Single Thread Small 4.255s 4.227s -0.6% ⚪ ✅
Excavate Single Thread Large 10.151s 10.096s -0.5% ⚪ ✅
Excavate Parallel Tasks Small 4.437s 4.452s +0.3% ⚪ ✅
Excavate Parallel Tasks Large 6.475s 6.535s +0.9% ⚪ ✅
Intercept Throughput Small 906.57ms 899.04ms -0.8% ⚪ ✅
Intercept Throughput Medium 944.28ms 896.07ms -5.1% ⚪ ✅
Dns Throughput Quiet 2.781s 2.854s +2.6% ⚪ ✅
Dns Throughput Loaded 1.951s 1.888s -3.2% ⚪ ✅
Dns Throughput Inherited 2.408s 2.504s +4.0% ⚪ ✅
Is Ip Performance 2.39ms 2.37ms -0.6% ⚪ ✅
Make Ip Type Performance 250.63µs 243.52µs -2.8% ⚪ ✅
Mixed Ip Operations 2.47ms 2.47ms -0.1% ⚪ ✅
Memory Use Web Crawl 1.2 MB 1.2 MB +0.0% ⚪ ✅
Memory Use Subdomain Enum 28.3 MB 28.3 MB +0.0% ⚪ ✅
Memory Use Deep Chain 5.0 MB 5.0 MB +0.0% ⚪ ✅
Memory Use Parallel Chains 13.4 MB 13.1 MB -2.8% ⚪ ✅
Scan Throughput 100 2.862s 2.978s +4.1% ⚪ ✅
Scan Throughput 1000 20.579s 21.951s +6.7% ⚪ ✅
Typical Queue Shuffle 5.48µs 5.42µs -1.1% ⚪ ✅
Priority Queue Shuffle 26.49µs 26.03µs -1.7% ⚪ ✅

🎯 Performance Summary

✅ No significant performance changes detected (all changes <10%)


🐍 Python Version 3.11.16

@liquidsec liquidsec self-assigned this Oct 1, 2026
@liquidsec liquidsec added this to the BBOT 3.1 - violent_barbara milestone Oct 1, 2026
@liquidsec
liquidsec merged commit b2dd81a into dev Oct 5, 2026
20 checks passed
@liquidsec
liquidsec deleted the lightfuzz-serial-error-resolution branch October 5, 2026 19:11
@liquidsec liquidsec mentioned this pull request Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants