BUG: sample StochasticFlight inputs once per simulation (#1090) - #1126
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #1126 +/- ##
========================================
Coverage 84.57% 84.57%
========================================
Files 131 131
Lines 17525 17527 +2
========================================
+ Hits 14821 14824 +3
+ Misses 2704 2703 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
I compared the flight inputs at current // develop
{
"flown": {"heading": 53.55816157603624, "inclination": 81.41474397857291, "rail_length": 5.2},
"logged": {"heading": 53.55816157603624, "inclination": 83.62213583873555, "rail_length": 5.2}
}
// PR head
{
"flown": {"heading": 52.78714695725011, "inclination": 85.16622077057734, "rail_length": 5.2},
"logged": {"heading": 52.78714695725011, "inclination": 85.16622077057734, "rail_length": 5.2}
}The absolute sampled values differ between implementations because the head intentionally makes one draw instead of three. The relevant invariant is that the values flown and recorded are identical within the same run; that fails on
Environment: Python 3.12.6; NumPy 2.5.2; pytest 9.1.1; macOS 26.5.2 arm64. |
|
I wonder whether this is changing the monte carlo behavior when we work with parallel mode |
os.replace is atomic for one file. Three of them were three atomic steps with nothing between, so a failure on the second left the first already replaced by an empty file, the untouched temporaries behind, and the destination narrowed from 0644 to the 0600 a staged file opens at. Each destination is now moved aside before its replacement goes in, anything already installed is put back if a later one fails, the temporaries that never landed are removed, and the mode of the log being replaced is carried onto the file replacing it. BaseException, so a Ctrl-C rolls back too. This is not a filesystem transaction and the docstring says so: a generation directory swapped by one pointer would be the stronger guarantee, and would change what the three public log paths mean. Also drops the RocketPy-Team#1090 fix from this branch. RocketPy-Team#1126 is open against the same function, adds the shared _sample_flight_inputs the fix wants, and closes the second draw inside StochasticFlight.create_object that this branch left alone. Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
os.replace is atomic for one file. Three of them were three atomic steps with nothing between, so a failure on the second left the first already replaced by an empty file, the untouched temporaries behind, and the destination narrowed from 0644 to the 0600 a staged file opens at. Each destination is now moved aside before its replacement goes in, anything already installed is put back if a later one fails, the temporaries that never landed are removed, and the mode of the log being replaced is carried onto the file replacing it. BaseException, so a Ctrl-C rolls back too. This is not a filesystem transaction and the docstring says so: a generation directory swapped by one pointer would be the stronger guarantee, and would change what the three public log paths mean. Also drops the RocketPy-Team#1090 fix from this branch. RocketPy-Team#1126 is open against the same function, adds the shared _sample_flight_inputs the fix wants, and closes the second draw inside StochasticFlight.create_object that this branch left alone. Signed-off-by: thc1006 <84045975+thc1006@users.noreply.github.com>
|
This is now conflicting with Also still open from my earlier comment, and it is the part I actually want pinned down before merging: what does "sample once per simulation" mean under parallel mode? |
Conflict was one test file: both sides appended to the end of tests/unit/stochastic/test_stochastic_flight.py, develop's RocketPy-Team#1109 tests for initial_solution and this branch's RocketPy-Team#1090 regressions. Kept both. One real interaction beyond the conflict. RocketPy-Team#1164, merged today, added test_a_monte_carlo_flight_keeps_the_configuration_it_was_given, which stubs self.flight with a types.SimpleNamespace carrying _randomize_rail_length, _randomize_inclination and _randomize_heading. __run_single_simulation now asks for _sample_flight_inputs instead, so the stub raised AttributeError. Updated it to the single-draw API and, since the stub now hands over all three values in one dict, made the test also assert they reach the Flight. That ties RocketPy-Team#1164's configuration test to RocketPy-Team#1090's invariant rather than leaving the sampled values unchecked. Confirmed the bug this fixes is a data-integrity one, not just wasted draws. On develop, __run_single_simulation called the three _randomize_* helpers and each ran its own next(dict_generator()), so a simulation flew the first draw's rail_length and the second draw's inclination while last_rnd_dict -- which is what gets written to the inputs file -- held only the third. With spreads on all three inputs and seed 4242: flown rail_length=5.433110 inclination=83.622136 heading=54.525483 logged rail_length=5.526158 inclination=86.327132 heading=54.525483 so every exported row has carried a rail length off by 9.3 cm and an inclination off by 2.7 degrees from the flight that produced the outputs. Only heading agreed, because it happened to be drawn last. After this change all three match. Also answered my own question on the PR about parallel mode: there is no hazard. __run_in_parallel gives each worker its own spawned seed, so consolidating three draws into one only reduces how many numbers a worker consumes and cannot couple workers. Measured: a fixed seed reproduces its draws, distinct seeds diverge, four worker seeds produce no repeated draw, and last_rnd_dict equals the returned draw on every call. Fixed-seed baselines do move, since a simulation now consumes one draw instead of three. That is the fix, not a side effect, but it is worth a changelog line. Verified: tests/unit 2163 passed, 17 skipped; ruff check and format clean; pylint 10.00/10. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Merged Answering my own earlier question about parallel mode: there is no hazard. The bug is worse than the title suggests, and worth saying out loud in the changelog. It isn't just wasted draws — it's a data-integrity bug. Every exported inputs row has carried a rail length off by 9.3 cm and an inclination off by 2.7 degrees from the flight that produced the outputs next to it. Only One real interaction I had to fix. #1164 merged today and added Two notes, neither blocking:
Verified before pushing: |
Pull request type
Checklist
black rocketpy/ tests//make lint) has passed locallypytest tests -m slow --runslow) have passed locallyCHANGELOG.md— no action needed; an LLM workflow auto-updates it after mergeCurrent behavior
MonteCarlo.__run_single_simulationcalled_randomize_rail_length,_randomize_inclination, and_randomize_heading, each of which invokednext(self.dict_generator()). One simulation therefore drew three times; the Flight mixed values across draws whilelast_rnd_dict(and.inputs.txt) kept only the last draw.StochasticFlight.create_objecthad the same pattern forrail_length.Fixes #1090
New behavior
StochasticFlight._sample_flight_inputs()performs a single draw.create_objectandMonteCarlo.__run_single_simulationboth use that helper so flown rail/inclination/heading matchlast_rnd_dict. The_randomize_*helpers remain for single-field callers and still each draw once.Breaking change
Additional information
Focused unit tests assert create_object and
__run_single_simulationFlight inputs equallast_rnd_dict. Full slow suite not run in this contribution pass.