Repository navigation
[No QA] [DO NOT MERGE] Reassure duration noise study - #106840
Draft
elirangoshen wants to merge 3 commits into
Draft
elirangoshen wants to merge 3 commits into
elirangoshen wants to merge 3 commits into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Explanation of Change
Do not merge. Temporary CI experiment; this PR will be closed once the run finishes.
Adds one temporary workflow,
reassureNoiseStudy.yml, that runs the existingtests/perf-localReassure function benchmarks six times on the same code, on 10 runners, and uploads the raw results as artifacts. No app code changes.The goal is to measure how much benchmark durations differ between identical runs on CI runners. That number decides whether Reassure durations can be used as an automated PR check, the way render counts are used today. A local run of the same study showed differences large enough to raise false alarms, so I need the CI number before proposing anything.
The workflow only triggers on changes to its own file, so it runs on this PR only.
Fixed Issues
$ #104376 (comment)
PROPOSAL:
Tests
Reassure duration noise study (temporary experiment)workflow run on this PRnoise-studyjobs finish and each uploads anoise-results-<n>artifact with sixpass-<i>.perffilesOffline tests
None. CI-only change.
QA Steps
None. CI-only experiment, not meant to be merged.
PR Author Checklist
### Fixed Issuessection aboveTestssectionOffline stepssectionQA stepssectionAvatar, I verified the components usingAvatarare working as expected)npm run compress-svg)Avataris modified, I verified thatAvataris working as expected in all cases)Designlabel and/or tagged@Expensify/designso the design team can review the changes.mainbranch was merged into this PR after a review, I tested again and verified the outcome was still expected according to theTeststeps.Screenshots/Videos
Android: Native
Android: mWeb Chrome
iOS: Native
iOS: mWeb Safari
MacOS: Chrome / Safari