Skip to content

Mutmut - #2278

Merged
amilcarlucas merged 15 commits into
masterfrom
mutmut
Oct 27, 2026
Merged

Mutmut#2278
amilcarlucas merged 15 commits into
masterfrom
mutmut

Conversation

@amilcarlucas

@amilcarlucas amilcarlucas commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator

Description

Add mutation testing

Checklist

  • Run pre-commit checks locally
  • Verified by a human programmer
  • All commits are signed off (use git commit --signoff)
  • Code follows our coding standards
  • Documentation updated if needed
  • No breaking changes or properly documented

Testing

Describe how you tested these changes:

  • Unit tests pass
  • Integration tests pass
  • Manual testing performed
  • Tested on flight controller hardware

@amilcarlucas
feat(mutmut): Add initial mutation test support using mutmut
bced819 
Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@amilcarlucas
test(mutation): Add missing survivor tests
c117753 
Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@amilcarlucas
test(mutation): expand suite and strengthen validation regressions
bc26da7 
Expand the two-module mutation pilot to 32 modules and improve test
selection and assertions for observable behavior.

- Cover parameter handling, vehicle components, firmware upload,
  bootloader communication, log analysis, AutoTune, and GUI workflows.
- Align the seven do_not_mutate exclusions with .coveragerc omissions.
- Include existing validation, schema, and constants tests previously
  missing from the mutation selection.
- Load plugin fixtures explicitly and copy test helpers and resources
  required by the selected tests into Mutmut's generated tree.
- Derive the CI clean baseline from pyproject.toml to keep test
  selection synchronized with mutation runs.
- Increase CI job and mutation-step limits to 240 and 230 minutes,
  respectively, and summarize mutation outcomes by module.
- Verify that diagnostics polling and banner reception use nonblocking
  MAVLink reads.
- Add seven voltage-correction cases that verify the conflicting field
  and persisted correction exactly 0.01 V beyond its limit.
- Add seven cases accepting equality at related voltage boundaries.
- Add two cases verifying repeated required-MCU schema updates preserve
  field metadata without reporting errors.
- Document selection audits, required resources, cache refreshes, and
  interpretation of mutation outcomes.

Validation:
- Completed the initial 32-module run across 24,697 mutants.
- Final selected clean baseline: 2,137 tests passed.
- Fresh validation/schema runs covered 2,422 mutants each.
- Corrected selection increased kills from 588 to 1,406.
- New regressions caught 36 additional survivors, yielding 1,442 killed,
  722 surviving, and 258 without associated tests.
- All 16 new regression cases passed normally and failed by assertion
  against their targeted mutations in isolated verification.
- Pre-commit, targeted pylint, and git diff --check passed.

Remaining unassociated validation mutants are in ESC telemetry helpers.
GUI timeout and segfault classifications remain unresolved.

Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@amilcarlucas
test(validation): cover ESC telemetry selection and protocol cascades
355bbe1 
Add 37 behavior cases using the real component model and schema to
verify ESC telemetry choices through public APIs.

- Check compatible ports and mirror state for PWM, DShot, CAN and
  same-port serial protocols across all four vehicle types.
- Distinguish scripting, ESC telemetry and mirrored serial protocols.
- Verify safe fallbacks for empty, stale and incompatible selections.
- Check CAN-to-PWM transitions and subsequent protocol changes for
  stale choices and duplicates.

Validation:
- All 2,174 selected tests pass.
- Complete all 1,914 validation mutants: 1,337 killed, 577 surviving,
  zero without tests, zero timeouts and zero segfaults.
- Catch 331 additional mutants and associate tests with all six
  previously uncovered ESC telemetry helpers.
- Confirm assertion failures for a representative mutation in each
  helper.
- Pre-commit, targeted pylint and git diff --check pass.

Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@amilcarlucas
test(validation): verify firmware metadata and offline choices
057dcfc 
Add 32 behavior cases using the real component model and schema to
verify choices presented by the component editor.

- Check complete initial connection-port lists and full protocol
  labels when metadata and saved connections are absent.
- Select FRAME_CLASS or Q_FRAME_CLASS according to vehicle type.
- Preserve Heli and Rover frame fallbacks when metadata is unusable,
  and retain Plane's valid Undefined-only frame choice.
- Apply loaded MOT_PWM_TYPE and Q_M_PWM_TYPE choices, preferring the
  QuadPlane parameter when both are available.
- Keep offline PWM choices when motor metadata is missing or empty.
- Support receiver enum and bitmask metadata, with nonempty enum
  values taking priority, and verify battery-monitor labels.
- Replace stale motor-protocol choices when metadata is reloaded.

Validation:
- All 191 validation-file tests and 2,206 selected tests pass.
- Complete all 1,914 validation mutants: 1,420 killed, 494 surviving,
  zero without tests, zero timeouts and zero segfaults.
- Catch 83 additional initialization mutants, reducing that function's
  survivors from 224 to 141 while preserving every previous kill.
- Verify assertion failures for seven representative mutations using
  the new cases alone.
- Pre-commit, targeted pylint and git diff --check pass.

Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@amilcarlucas
test(validation): cover saved connection initialization
6954c9d 
Add 23 behavior cases that load persisted component data through the
real model initialization workflow without calling connection setters.

- Rebuild receiver, telemetry, GNSS and battery-monitor protocol choices
  from saved connection types while preserving their stored settings.
- Cover disabled connections and analog, I2C, CAN, PWM, SPI and virtual
  battery-monitor configurations.
- Verify complete initial battery-monitor and ESC selectors, excluding
  internal mirror markers and duplicate telemetry protocols.
- Rebuild saved PWM, DShot, CAN and same-port serial ESC telemetry
  choices, including explicitly disabled telemetry.
- Verify legacy ESC connection migration retains control settings and
  initializes disabled telemetry with compatible port choices.

Validation:
- All 214 validation-file tests and 2,229 selected tests pass.
- Complete all 1,914 validation mutants: 1,556 killed, 358 surviving,
  zero without tests, zero timeouts and zero segfaults.
- Catch 136 additional mutants while preserving every previous kill.
- Reduce initialization survivors from 141 to 29 and catch additional
  ESC telemetry, battery-monitor and GNSS filtering mutations.
- Verify assertion failures for nine representative startup mutations
  using the new cases alone.
- Pre-commit, targeted pylint and git diff --check pass.

Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@amilcarlucas
test(validation): cover component batch conflicts and error reporting
2541586 
Add 22 behavior cases using real component models and public APIs to
verify cross-component validation without mocks or private-state edits.

- Reject GNSS serial-port conflicts with RC receivers, telemetry, ESC
  control and ESC telemetry in either field order.
- Accept RC/telemetry and bidirectional ESC serial sharing while still
  reporting later independent port conflicts and motor-pole errors.
- Accept shared CAN1/CAN2 buses and multiple disabled peripherals.
- Check invalid selections report the rejected value, full field path
  and allowed choices before continuing through the remaining fields.
- Validate CAN2/DroneCAN type and protocol selections together without
  treating repeated protocol names as exclusive physical ports.
- Assert complete error lists so early exits cannot hide later errors.

Validation:
- All 236 validation-file tests and 2,251 selected tests pass.
- Complete all 1,914 validation mutants: 1,593 killed, 321 surviving,
  zero without tests, zero timeouts and zero segfaults.
- Catch 37 additional mutants while preserving every previous kill.
- Reduce validate_all_data survivors from 43 to 10, with additional
  kills in motor-pole validation and numeric error propagation.
- Verify assertion failures for 12 representative prior survivors
  using the new cases alone, with zero setup errors.
- Ruff, targeted pylint, pre-commit and git diff --check pass.

Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@amilcarlucas
test(validation): cover ESC telemetry during connection transitions
7b31bb4 
Add 27 behavior cases using real component models and public APIs to
check telemetry selectors immediately after each control-port change.

- Preserve DroneCAN telemetry while moving ESCs between CAN1 and CAN2.
- Move bidirectional serial ESCs to SERIAL1 or SERIAL8 while retaining
  the mirrored FETtecOneWire, Torqeedo or CoDevESC protocol.
- Exclude stale CAN/PWM protocols when switching to serial, then refresh
  telemetry after selecting a compatible control protocol.
- Clear prior telemetry choices when the control connection is disabled.
- Keep new PWM modes from firmware metadata selectable while offering
  only disabled telemetry when no built-in compatibility mapping exists.
- Cover Main Out/AIO across Copter, Heli, Rover and Plane, including
  Q_M_PWM_TYPE metadata for Plane.

Assert both telemetry selectors before a later field edit can overwrite
their values. Use no mocks or private-state edits and leave existing
tests, production code and mutation configuration unchanged.

Validation:
- All 263 validation-file tests and 2,278 selected tests pass.
- Complete all 1,914 validation mutants: 1,626 killed, 288 surviving,
  zero without tests, zero timeouts and zero segfaults.
- Catch 33 additional mutants while retaining every previous kill.
- Reduce general ESC telemetry selector survivors from 31 to 5 and
  PWM mapping survivors from 33 to 26.
- Verify assertion failures for ten representative prior survivors
  using the new cases alone, with zero setup errors.
- Ruff, targeted pylint, pre-commit and git diff --check pass.

Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@amilcarlucas
test(validation): verify numeric limits and takeoff-weight corrections
e080f58 
Add 38 behavior cases using real component models and public APIs to
verify numeric guidance, inclusive boundaries and applied corrections.

- Cover both endpoints and out-of-range values for takeoff weights,
  battery cell count/capacity, motor poles and propeller diameter.
- Check range errors include the field, required numeric type and both
  limits, and conversion failures preserve the existing field value.
- Verify entry validation previews the nearest allowed value without
  changing data and batch validation applies that exact correction.
- Check crossed takeoff weights identify the opposing field and value,
  correct by 0.01 kg and preserve the unedited endpoint.
- Load malformed related weights through saved-data initialization and
  report invalid types or nonnumeric text without replacing either value.
- Exercise battery-voltage cross-field constraints through the general
  entry API and confirm the same correction is applied during a batch.
- Revalidate corrected values to confirm the proposed repair is valid.

Use no mocks or private-state edits and leave previously committed tests,
production code and mutation configuration unchanged.

Validation:
- All 301 validation-file tests and 2,316 selected tests pass.
- Complete all 1,914 validation mutants: 1,658 killed, 256 surviving,
  zero without tests, zero timeouts and zero segfaults.
- Catch 32 additional mutants while retaining every previous kill.
- Reduce entry-limit survivors from 22 to 10, takeoff-weight survivors
  from 19 to 9 and shared comparison survivors from 8 to zero.
- Catch two additional battery-voltage mutations.
- Verify assertion failures for 14 representative prior survivors
  using the new cases alone, with zero setup errors.
- Ruff, targeted pylint, pre-commit and git diff --check pass.

Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@amilcarlucas
test(validation): verify battery voltage guidance and corrections
3346b4c 
Add 56 behavior cases using real component models and public APIs to
verify chemistry limits, voltage relationships and invalid-input recovery.

- Accept exact absolute minimum and maximum voltages for all eight
  supported chemistries and check the guidance for out-of-range values.
- Verify errors identify the chemistry, exact limit or related threshold
  so generic error assertions cannot hide incorrect diagnostic details.
- Cover all seven cross-field voltage relationships and preserve the
  other thresholds when batch validation corrects the edited field.
- Check invalid text, empty values and decimal-comma text for all five
  voltage thresholds, including the recovery instruction and exact
  recommended value applied by batch validation.
- Verify preview validation preserves data and corrected values pass
  subsequent entry validation.
- Keep unrelated battery capacity and product fields outside the scope
  of the dedicated cell-voltage validator.

Avoid dependence on Python conversion-error wording. Use no mocks or
private-state edits and leave existing tests, production code and the
mutation configuration unchanged.

Validation:
- All 357 validation-file tests and 2,372 selected tests pass.
- Complete all 1,914 validation mutants: 1,675 killed, 239 surviving,
  zero without tests, zero timeouts and zero segfaults.
- Catch 17 additional mutants while retaining every previous kill.
- Reduce validate_cell_voltage survivors from 21 to four; individually
  review the remaining False-to-None argument changes as equivalent for
  the current comparison helper without adjusting the raw results.
- Verify assertion failures for eleven representative prior survivors
  using the new cases alone, with zero setup errors.
- Ruff, targeted pylint, pre-commit and git diff --check pass.

Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@amilcarlucas
test(schema): cover metadata fallbacks and composed fields
6a754a9 
Add 50 behavior cases exercising the schema editor's public APIs with
representative metadata dictionaries, without mocks or private calls.

- Check exact datatype extraction for properties composed with allOf,
  including empty placeholders and unknown-type string fallback.
- Handle a Components node without property definitions safely.
- Distinguish quiet defaults for missing optional metadata from caught
  lookup exceptions that previously returned the same default tuple.
- Keep unrelated product and allOf section properties from triggering
  absent-field lookups or spurious error logs.
- Follow reference aliases to direct or composed nested field metadata
  while preserving the description and optional flag.
- Verify missing MCU schema diagnostics identify the unavailable
  structure for both required and optional update requests.
- Preserve malformed schema data and retain useful operation/type
  context in MCU-update and description-lookup error messages.

Leave existing tests, production code and mutation configuration unchanged.

Validation:
- Establish a fresh baseline matching the earlier 508 schema outcomes:
  436 killed and 72 surviving, with zero execution errors.
- Complete a fresh regression run: 488 killed and 20 surviving, with
  zero without tests, zero timeouts and zero segfaults.
- Catch 52 additional mutants while retaining every previous kill.
- All 67 schema-file tests and 2,422 selected tests pass.
- Verify assertion failures for eleven representative prior survivors
  using the new cases alone, with zero setup errors.
- Individually review eight equivalents without adjusting raw results;
  keep twelve MCU debug-message mutations as unresolved survivors.
- Ruff, targeted pylint, pre-commit and git diff --check pass.

Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@amilcarlucas
test(components): cover legacy battery loading and typed values
e2794e9 
Add 40 behavior cases using real schemas, models and parameter dictionaries
through public APIs without mocks or private-state edits.

- Initialize legacy battery settings with controller, file, mixed or
  absent voltage parameters available before migration runs.
- Verify controller precedence, per-parameter fallback, single-cell and
  legacy-text counts, and pack-to-cell rounding to four decimals.
- Use chemistry defaults for invalid, zero or negative cell counts.
- Preserve existing thresholds, custom scalar/nested data and original
  input dictionaries while ordering newly migrated fields canonically.
- Retain complete settings and their field order when reopening with
  different controller parameters.
- Store supported Boolean spellings as actual bool values and preserve
  objects already matching the schema's dictionary type.
- Retain unconvertible text with warnings that identify the value,
  target datatype, component path and structured-data requirement.

Leave existing tests, production code and mutation configuration unchanged.

Validation:
- Establish a fresh 867-mutant baseline: 641 killed and 226 surviving.
- Complete a fresh regression run: 725 killed and 142 surviving, with
  zero without tests, zero timeouts and zero segfaults.
- Catch 84 additional mutants while retaining every previous kill.
- Reduce cast-helper survivors from 46 to eight and battery-migration
  survivors from 39 to seven, with further initialization/setter gains.
- All 118 base-file tests and 2,462 selected tests pass.
- Verify assertion failures for 14 representative prior survivors
  using the new cases alone, with zero setup errors.
- Record eleven individual equivalence reviews without adjusting the
  raw results; distinguish 28 earlier gains from this batch's 84 gains.
- Ruff, targeted pylint, pre-commit and git diff --check pass.

Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@amilcarlucas
test(components): cover saved structure defaults and ordering
8c5b6b3 
Add 15 BDD cases using real schemas and the public project-loading API:

- Verify exact format, program, controller, frame and battery defaults.
- Preserve saved values, explicit zero and nested project extensions while
  refreshing the program version and leaving caller-owned input untouched.
- Check the full standard component sequence and custom component order.
- Cover optional product identity fields, version/URL ordering and partial
  metadata preservation, including Notes-last behavior.
- Exercise legacy ESC connection merging, telemetry preservation/defaults,
  and GNSS component/protocol renaming without losing custom settings.
- Confirm loaded projects remain stable when initialized again.

Verify the same 867 base-model mutants: 792 killed and 75 surviving,
catching 67 additional mutants while retaining all 725 previous kills.
Confirm the new cases alone fail by assertion under 14 formerly surviving
mutations and pass against unmodified generated source.

Validation: 2,477 configured tests pass, including all 133 base-file tests;
Ruff format/check, Pylint, pre-commit and git diff --check pass.
Production code, existing tests and mutation configuration are unchanged.

Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@amilcarlucas
test(mutation): strengthen behavior coverage and audit test selection
8ec0c44 
Expand regression coverage for observable model and GUI behavior, and
include existing suites that were missing from Mutmut's test selection.

- Select manual-override unit, acceptance, and integration tests, plus
  derived-parameter confirmation tests.
- Cover IMU magnitudes across axes, all supported orientations, detection
  boundaries, scale invariance, and polling callback wiring.
- Verify every supported AutoTune result filename and ensure aliases
  reduce only the intended axis gains.
- Exercise inclusive parameter limits, independent FC-value validation,
  value resets, snapshot copy independence, and tolerance boundaries.
- Check battery cell-count limits, invalid-voltage fallbacks, chemistry
  detection priority, and known firmware protocol renames.
- Verify upload cancellation, cancelled and invalid renames, rejected
  motor settings, safety confirmation refusal, and timed status feedback.
- Control motor-view timers before construction, destroy views during
  teardown, and mock informational dialogs.
- Document equivalent mutants, focused reruns, result preservation, and
  fork-server isolation diagnostics.

Validation:
- Pass all 2,693 clean tests in the expanded mutation selection.
- Confirm 352 additional kills across 1,262 targeted mutants.
- Complete a separate 3,886-mutant baseline refresh, including coverage
  improvements from earlier commits.
- Confirm four parameter worker failures as assertion kills with
  fork-server preloading disabled; retain five sampled GUI timeouts.
- Pass pre-commit and Pylint checks.

Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@github-actions

github-actions Bot commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

☂️ Code Coverage

current status: ✅

Overall Coverage

Statements Covered Coverage Threshold Status
29623 27694 93% 89% 🟢

New Files

No new covered files...

Modified Files

No covered modified files...

updated for commit: 84ed73b by action🐍

@github-actions

github-actions Bot commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Test Results

     3 files   -     1       3 suites   - 1   43m 21s ⏱️ - 23m 16s
 8 452 tests +  544   8 443 ✅ +  542   5 💤 + 2  4 ❌ +1 
25 224 runs   - 6 200  25 192 ✅  - 6 173  28 💤  - 27  4 ❌ +1 

For more details on these failures, see this check.

Results for commit 84ed73b. ± Comparison against base commit cc1d26f.

This pull request removes 10 and adds 554 tests. Note that renamed tests count towards both. 
tests.plugins.test_imu_helpers ‑ test_compute_detected_position[sample1-RIGHT]
tests.plugins.test_imu_helpers ‑ test_compute_detected_position[sample2-INDEFINITE]
tests.plugins.test_imu_helpers ‑ test_compute_detected_position[sample3-INDEFINITE]
tests.plugins.test_imu_helpers ‑ test_compute_movement_magnitude_converts_milligravity_to_metres_per_second_squared
tests.plugins.unit_data_model_battery_monitor ‑ test_user_sees_continuous_readings_from_round_robin_battery_telemetry[16]
tests.plugins.unit_data_model_battery_monitor ‑ test_user_sees_continuous_readings_from_round_robin_battery_telemetry[1]
tests.plugins.unit_data_model_battery_monitor ‑ test_user_sees_continuous_readings_from_round_robin_battery_telemetry[2]
tests.plugins.unit_data_model_battery_monitor ‑ test_user_sees_continuous_readings_from_round_robin_battery_telemetry[9]
tests.unit_battery_cell_voltages.TestBatteryCell ‑ test_best_chemistry_for_voltage_identifies_chemistry_for_4s_voltage
tests.unit_battery_cell_voltages.TestBatteryCell ‑ test_best_chemistry_for_voltage_returns_none_or_valid_chemistry_for_ambiguous_voltage

tests.bdd_frontend_tkinter_file_browser.TestFileBrowserUserWorkflows ‑ test_cancelled_or_invalid_rename_preserves_files_and_restores_navigation[-local]
tests.bdd_frontend_tkinter_file_browser.TestFileBrowserUserWorkflows ‑ test_cancelled_or_invalid_rename_preserves_files_and_restores_navigation[-remote]
tests.bdd_frontend_tkinter_file_browser.TestFileBrowserUserWorkflows ‑ test_cancelled_or_invalid_rename_preserves_files_and_restores_navigation[../outside.bin-local]
tests.bdd_frontend_tkinter_file_browser.TestFileBrowserUserWorkflows ‑ test_cancelled_or_invalid_rename_preserves_files_and_restores_navigation[../outside.bin-remote]
tests.bdd_frontend_tkinter_file_browser.TestFileBrowserUserWorkflows ‑ test_cancelled_or_invalid_rename_preserves_files_and_restores_navigation[None-local]
tests.bdd_frontend_tkinter_file_browser.TestFileBrowserUserWorkflows ‑ test_cancelled_or_invalid_rename_preserves_files_and_restores_navigation[None-remote]
tests.bdd_frontend_tkinter_file_browser.TestFileBrowserUserWorkflows ‑ test_declining_an_upload_preserves_remote_bytes_and_creates_no_progress_window
tests.plugins.test_autotune_gain_backoff ‑ test_each_supported_result_filename_selects_exactly_the_tuned_axes[pitch-expected_axes1]
tests.plugins.test_autotune_gain_backoff ‑ test_each_supported_result_filename_selects_exactly_the_tuned_axes[pitch_RatePD-expected_axes6]
tests.plugins.test_autotune_gain_backoff ‑ test_each_supported_result_filename_selects_exactly_the_tuned_axes[pitch_rate_pd-expected_axes10]
…
This pull request skips 2 tests. 
tests.gui_frontend_tkinter_show.TestTooltipUserExperienceIntegration ‑ test_gui_functionality_on_macos
tests.gui_frontend_tkinter_show.TestTooltipUserExperienceIntegration ‑ test_tooltip_positioning_with_retina_display

♻️ This comment has been updated with latest results.

@amilcarlucas amilcarlucas added the AIReview Request an automated AI review; picked up by the reviewprs sweep label Oct 26, 2026
@AP-Review

AP-Review commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown

Deprecated — see below for the updated review.

Previous review (2026-10-26)

Automated review note — AI-generated (Claude+Codex), validated against the live diff. Please sanity-check before acting.
Verdict: REQUEST CHANGES

Reviewed at head 8ec0c44ce6.
Full report: https://firmware.ardupilot.org/Tools/APReview/DevCallReviews/PRReviews/ardupilot/methodicconfigurator/2278/1.html#prMethodicConfigurator-2278

This PR adds Mutmut mutation testing and strengthens many tests, but the pylint CI job fails on duplicate code introduced by the new tests.

Blocking

  1. tests/test_data_model_vehicle_components_base.py:1726: EXPECTED_ORDER and the expected component tuple at line 2116 copy production lists verbatim. Pylint R0801 then fails the pylint (3.14) job with exit 8. Wrap both in # pylint: disable=duplicate-code / # pylint: enable=duplicate-code, as tests/test_data_model_vehicle_components_json_schema.py already does.

Minor

  • .github/workflows/mutation.yml:69: the comment, and CONTRIBUTING.md:137, say execution errors fail the job. mutmut 3.8.0 exits 0 when mutants time out or segfault, so reword to say clean-test and framework failures fail the job and per-mutant timeouts and crashes are reported for review, which matches CONTRIBUTING.md:146.

Withdrawn: the concern that the sub-volt chemistry boundary test locks in degenerate behaviour. It tests the helper's documented threshold.

@amilcarlucas
test(mutation): expand coverage to six additional model modules
84ed73b 
Expand Mutmut's scope from 32 to 38 modules and select their existing
focused test suites. Add 72 test cases targeting observable behavior
exposed by surviving mutants.

- Cover flight-controller frame fallbacks, throttle and duration limits,
  and default motor sequence numbering.
- Verify one-byte transfers, estimates at exactly one second, and
  transitions between known and unknown file sizes.
- Check every supported configuration expression helper.
- Exercise firmware migration boundaries, collision diagnostics,
  invalid version inputs, and numeric serial-port ordering.
- Cover secondary-battery thresholds, unavailable readings, reconnect
  and re-enable recovery, and telemetry subscription retries.
- Assert expected telemetry intervals before advancing the round-robin
  simulation so zero-interval mutants fail immediately.
- Verify configuration-step metadata, defaults, forced and derived
  values, manual overrides, live expression inputs, deletion priority,
  auto-import identity, connection conflicts, and ExpressLRS guidance.
- Add scoped pylint duplicate-code suppressions around independent
  expected ordering tuples.
- Document equivalent and unreachable mutation paths, and clarify that
  per-mutant timeouts and crashes are reported for review while
  clean-test and framework failures fail the workflow.

Validation:
- 3,001 configured tests pass.
- Targeted kills increase from 792 to 976 across 1,109 mutants.
- No unresolved targeted timeouts, crashes, or missing test associations.
- Repository-wide pylint and pre-commit checks pass.

Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@AP-Review

AP-Review commented Oct 27, 2026

Copy link
Copy Markdown

Automated review note — AI-generated (Claude+Codex), validated against the live diff. Please sanity-check before acting.
Verdict: ACCEPT

Reviewed at head 84ed73b4b7.
Full report: https://firmware.ardupilot.org/Tools/APReview/DevCallReviews/PRReviews/ardupilot/methodicconfigurator/2278/2.html#prMethodicConfigurator-2278

No open issues remain at head 84ed73b. This PR adds mutmut mutation-testing support and strengthens tests, with no production code changes.

Since the last review at 8ec0c44, both earlier findings are fixed. The duplicated expected tuples are now wrapped in scoped duplicate-code suppressions (tests/test_data_model_vehicle_components_base.py:1727, :2120), and pylint passes again. The failure wording in .github/workflows/mutation.yml:69 and CONTRIBUTING.md:140 now matches mutmut 3.8.0's exit behaviour.

The remaining red CI jobs are not caused by this PR: Windows layout failures that also fail on base, a network timeout on Ubuntu, a Homebrew setup failure on macOS, and a broken link in files the PR does not touch. The full mutation campaign and the GitHub run of the mutation workflow were not exercised here.

@amilcarlucas
amilcarlucas merged commit 0b41aca into master Oct 27, 2026
28 of 34 checks passed
@amilcarlucas
amilcarlucas deleted the mutmut branch October 27, 2026 02:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

AIReview Request an automated AI review; picked up by the reviewprs sweep

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@amilcarlucas @AP-Review