Add relative percentile degradation and error count ceiling criteria to AutoStop listener - #883
Conversation
undera
left a comment
There was a problem hiding this comment.
Thanks — this targets the right place, and the build is green here too (40 tests, 0 failures). I confirmed the intended percentile path works: rank 90, threshold 50%, 100ms -> 1000ms stops the test as designed.
Two blockers before this can go in, both left as inline comments:
relWindowStatCalc.addSample()runs on the per-sample path without synchronization, in aNoThreadClonelistener shared across all threads.- A blank percentile rank does not disable the new criterion — it runs it against P0 and can stop the test. The test that covers that configuration asserts nothing and its comment claims the opposite.
The rest of the inline notes are smaller: three assert-free tests, an undeclared change to the existing percentile fallback, the removed custom-validation row, and some wording and guard mismatches.
One doc item not tied to a line: site/dat/wiki/AutoStop.md still says "Currently 3 criteria available". It was already behind by one (percentile), and this makes six — good moment to bring it up to date, including the window semantics for both new criteria.
Version bump and the marketplace versions entry I will handle at release time; no need to touch those here.
…ation and error count checks according to review comments
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #883 +/- ##
============================================
+ Coverage 68.90% 69.00% +0.10%
- Complexity 2629 2702 +73
============================================
Files 230 233 +3
Lines 15965 16399 +434
Branches 1638 1710 +72
============================================
+ Hits 11000 11316 +316
- Misses 4146 4214 +68
- Partials 819 869 +50 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Description
As discussed in #880, this PR adds the missing criteria to the Auto-Stop Listener (
jpgc-autostop):Relative Window-to-Window Percentile Degradation:
90), max growth%(e.g.50%), observation window in seconds (e.g.5s).Per-Window Absolute Error Count Ceiling:
GUI Checkboxes & Layout:
Input Safety & Error Handling:
NumberFormatExceptionwhen fields are left blank or unconfigured.Examples & Unit Tests:
AutoStopTestcovering threshold breaches, window resets, and disabled states..jmxtest plans underexamples/:AutoStopExample_Percentile.jmxAutoStopExample_ErrCount.jmxVerification
mvn clean test packageruns with 0 failures, 0 errors across all 40 tests.