You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Due for payment]$125 Flaky Tests - GroupChatNameTests #65357
// Return a Promise that resolves with the mocked response
returnPromise.resolve(mockedResponse);
});
// Simulate user entering their login and populating the credentials.login
Session.beginSignIn(login);
returnwaitForBatchedUpdates()
.then(()=>{
HttpUtils.xhr=jest.fn().mockImplementation(()=>{
constmockedResponse: OnyxResponse={
onyxData: [
{
onyxMethod: Onyx.METHOD.MERGE,
key: ONYXKEYS.SESSION,
value: {
authToken,
accountID,
email: login,
encryptedAuthToken: authToken,
},
},
{
onyxMethod: Onyx.METHOD.MERGE,
key: ONYXKEYS.CREDENTIALS,
value: {
autoGeneratedLogin: Str.guid('expensify.cash-'),
autoGeneratedPassword: Str.guid(),
},
},
{
onyxMethod: Onyx.METHOD.MERGE,
key: ONYXKEYS.ACCOUNT,
value: {
isUsingExpensifyCard: false,
},
},
{
onyxMethod: Onyx.METHOD.MERGE,
key: ONYXKEYS.BETAS,
value: ['all'],
},
{
onyxMethod: Onyx.METHOD.MERGE,
key: ONYXKEYS.NVP_PRIVATE_PUSH_NOTIFICATION_ID,
value: 'randomID',
},
],
jsonCode: 200,
};
// Return a Promise that resolves with the mocked response
returnPromise.resolve(mockedResponse);
});
Session.signIn(password);
returnwaitForBatchedUpdates();
})
.then(()=>{
HttpUtils.xhr=originalXhr;
});
}
The problem comes from the previous test. We revert the mock after calling SignInUser API, which causes other APIs on the render App to fail because fetch is not defined. And the retry logic will call the API a few times. Then, when we go to the next test, the SignInUser is added to the queue and it can be called after the mock is reverted since it's the last API in the queue.
What changes do you think we should make in order to solve the problem?
We should set the fetch with our mock fetch to ensure this is always available. Use the fetch mock to ensure all APIs are success, then we don't have the problem SignInUser is called after we revert the mock.
What specific scenarios should we cover in automated tests to prevent reintroducing this issue in the future?
None
What alternative solutions did you explore? (Optional)
Reminder: Please use plain English, be brief and avoid jargon. Feel free to use images, charts or pseudo-code if necessary. Do not post large multi-line diffs or write walls of text. Do not create PRs unless you have been hired for this job.
I think the root cause of #59729 isn't clear, and the test is still flaky. So we need to dig deeper to find the real root cause and explore other solutions.
Thanks for the proposal, @nkdengineer! It looks promising and seems to fix the flaky tests.
Could you explain more about the root cause? Without your fix, I see the "fetch is not defined" message even when the test passes. So how could that cause this issue?
Activity
melvin-bot commentedon Jul 2, 2025
Triggered auto assignment to Contributor-plus team member for initial proposal review - @QichenZhu (
External)melvin-bot commentedon Jul 2, 2025
Triggered auto assignment to @Christinadobrzyn (
Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details. Please add this bug to a GH project, as outlined in the SO.nkdengineer commentedon Jul 3, 2025
Proposal
Please re-state the problem that we are trying to solve in this issue.
What is the root cause of that problem?
The problem is our mock
SignInUserAPI returns an errorfetch is not definedso the app doesn't render the sidebarApp/src/libs/HttpUtils.ts
Line 57 in 5b2b9c6
The reason is that the mock was reverted before
SignInUserAPI is called.App/tests/utils/TestHelper.ts
Lines 90 to 180 in 4a5b141
The problem comes from the previous test. We revert the mock after calling
SignInUserAPI, which causes other APIs on the render App to fail becausefetch is not defined. And the retry logic will call the API a few times. Then, when we go to the next test, theSignInUseris added to the queue and it can be called after the mock is reverted since it's the last API in the queue.What changes do you think we should make in order to solve the problem?
We should set the fetch with our mock fetch to ensure this is always available. Use the fetch mock to ensure all APIs are success, then we don't have the problem
SignInUseris called after we revert the mock.App/tests/ui/GroupChatNameTests.tsx
Line 127 in 5b2b9c6
What specific scenarios should we cover in automated tests to prevent reintroducing this issue in the future?
None
What alternative solutions did you explore? (Optional)
Reminder: Please use plain English, be brief and avoid jargon. Feel free to use images, charts or pseudo-code if necessary. Do not post large multi-line diffs or write walls of text. Do not create PRs unless you have been hired for this job.
QichenZhu commentedon Jul 3, 2025
I think the root cause of #59729 isn't clear, and the test is still flaky. So we need to dig deeper to find the real root cause and explore other solutions.
nkdengineer commentedon Jul 3, 2025
@QichenZhu How can you reproduce it locally?
QichenZhu commentedon Jul 3, 2025
@nkdengineer I ran the command below repeatedly, and it failed on the 4th run:
nkdengineer commentedon Jul 3, 2025
@QichenZhu Thanks, I can reproduce it locally now. Updated proposal
QichenZhu commentedon Jul 4, 2025
Thanks for the proposal, @nkdengineer! It looks promising and seems to fix the flaky tests.
Could you explain more about the root cause? Without your fix, I see the "fetch is not defined" message even when the test passes. So how could that cause this issue?
nkdengineer commentedon Jul 4, 2025
@QichenZhu Do you mean you see this message when all tests are passed or only some tests are passed.
29 remaining items
Christinadobrzyn commentedon Jul 14, 2025
PR = #66032 is merged
@nkdengineer are we waiting on others or can this follow our payment process? Is this $50 too?
nkdengineer commentedon Jul 19, 2025
@Christinadobrzyn No we don't need to wait on others issues. It's not related
Is this $50 too?
I think the price is the base price $250. Is that right @dangrous ?
dangrous commentedon Jul 22, 2025
So it looks like the recent benchmark for flaky tests is $125, let's use that
[-]Flaky Tests - GroupChatNameTests[/-][+][Due for payment]Flaky Tests - GroupChatNameTests[/+]Christinadobrzyn commentedon Jul 23, 2025
Payment Summary
Contributor: @nkdengineer paid $125 via Upwork (offer - https://www.upwork.com/nx/wm/offer/108158810)
Contributor+: @QichenZhu due $125 via Upwork (offer - https://www.upwork.com/nx/wm/offer/108158817)
Is this correct? @QichenZhu can you please provide your Upwork if the above it true?
QichenZhu commentedon Jul 23, 2025
Here's my upwork profile. Thanks!
nkdengineer commentedon Jul 23, 2025
@Christinadobrzyn, I should be paid via Upwork (my profile is https://www.upwork.com/freelancers/~0134d0b74faef1a5ac). Please help to check again and correct me if I missed something.
[-][Due for payment]Flaky Tests - GroupChatNameTests[/-][+][Due for payment]$125 Flaky Tests - GroupChatNameTests[/+]Christinadobrzyn commentedon Jul 23, 2025
Thanks @QichenZhu and @nkdengineer - can you please accept me Upwork offer here - #65357 (comment)
Thanks!
Christinadobrzyn commentedon Jul 24, 2025
Thanks @QichenZhu and @nkdengineer you both have been paid in Upwork - summary here - #65357 (comment)
Closing!