Repository navigation
[HOLD for payment 2023-11-21] [HOLD for payment 2023-11-09] [HOLD for payment 2023-11-01] Improve the reliability of the E2E testing suite #29660
Description
Problem
Currently, the E2E testing suite runs the tests on one device over and over. This leads to performance degradation as runs keep on coming since the devices on AWS do not do a complete clean reboot between the runs. This is making the results less accurate and hard to make any calls based upon.
Solution
Explore options how to structure and run these tests so they are more accurate.
Activity
ospfranco commentedon Oct 16, 2023
.
[-]Improve the reliability of the E2E testing suite[/-][+][HOLD for payment 2023-11-01] Improve the reliability of the E2E testing suite[/+]melvin-bot commentedon Oct 25, 2023
The solution for this issue has been 🚀 deployed to production 🚀 in version 1.3.90-2 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:
If no regressions arise, payment will be issued on 2023-11-01. 🎊
After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.
For reference, here are some details about the assignees on this issue:
As a reminder, here are the bonuses/penalties that should be applied for any External issue:
5 remaining items
[-][HOLD for payment 2023-11-09] [HOLD for payment 2023-11-01] Improve the reliability of the E2E testing suite[/-][+][HOLD for payment 2023-11-21] [HOLD for payment 2023-11-09] [HOLD for payment 2023-11-01] Improve the reliability of the E2E testing suite[/+]melvin-bot commentedon Nov 14, 2023
Reviewinglabel has been removed, please complete the "BugZero Checklist".melvin-bot commentedon Nov 14, 2023
The solution for this issue has been 🚀 deployed to production 🚀 in version 1.3.98-5 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:
If no regressions arise, payment will be issued on 2023-11-21. 🎊
After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.
For reference, here are some details about the assignees on this issue:
AndrewGable commentedon Nov 30, 2023
@hannojg is working on this!
hannojg commentedon Nov 30, 2023
Yeah i think we nailed this now and the tests are stable 😇🎉