Skip to content

[Network] Messages are sent twice  #4214

Closed

Description

@isagoico

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!


Action Performed:

  1. Log in to New Expensify with an unstable internet connection
  2. Send some messages to an account you have access too. (Preferably you're logged in on another device with)

Expected Result:

Messages sent with an unstable internet connection should be sent without any issues.

Actual Result:

Messages are sent twice when there's unstable internet connection

Workaround:

Unknown.

Platform:

Where is this issue occurring?

  • Web
  • iOS
  • Android
  • Desktop App
  • Mobile Web

Version Number: 1.0.80-0

Logs: https://stackoverflow.com/c/expensify/questions/4856

Notes/Photos/Videos: Any additional supporting documentation

image

image

Expensify/Expensify Issue URL:

View all open jobs on Upwork


From @JmillsExpensify https://expensify.slack.com/archives/C01GTK53T8Q/p1626890348078000

When chatting with someone on a slow wifi connection, you get two of every message they send.

Activity

  1. MelvinBot commented on Jul 24, 2021

    @MelvinBot
    Contributor

    Triggered auto assignment to @thienlnam (Engineering), see https://stackoverflow.com/c/expensify/questions/4319 for more details.

  2. added
    ExternalAdded to denote the issue can be worked on by a contributor
    on Jul 26, 2021
  3. MelvinBot commented on Jul 26, 2021

    @MelvinBot
    Contributor

    Triggered auto assignment to @JmillsExpensify (External), see https://stackoverflow.com/c/expensify/questions/8582 for more details.

  4. JmillsExpensify commented on Jul 28, 2021

    @JmillsExpensify
    Contributor

    Posted to Upwork here! https://www.upwork.com/jobs/~018c22cb61e93a114c. Make sure to share your technical proposal with us here and a colleague will approve your technical proposal. Thank you!

  5. added
    Help WantedApply this label when an issue is open to proposals by contributors
    on Jul 28, 2021
  6. Santhosh-Sellavel commented on Jul 28, 2021

    @Santhosh-Sellavel
    Collaborator

    Proposal

    The issue happens when the network is offline, retryableRequests are set in ONYX, but not cleared from networkRequestQueue(current queue)

    App/src/libs/Network.js

    Lines 105 to 117 in ad77386

    // NetInfo tells us whether the app is offline
    if (isOffline) {
    if (!networkRequestQueue.length) {
    return;
    }
    // If we have a request then we need to check if it can be persisted in case we close the tab while offline
    const retryableRequests = _.filter(networkRequestQueue, request => (
    !request.data.doNotRetry && request.data.persist
    ));
    Onyx.set(ONYXKEYS.NETWORK_REQUEST_QUEUE, retryableRequests);
    return;
    }

    After updating in the ONYX, here queue gets duplicated,

    App/src/libs/Network.js

    Lines 32 to 45 in ad77386

    Onyx.connect({
    key: ONYXKEYS.NETWORK_REQUEST_QUEUE,
    callback: (persistedRequests) => {
    if (didLoadPersistedRequests || !persistedRequests) {
    return;
    }
    // Merge the persisted requests with the requests in memory then clear out the queue as we only need to load
    // this once when the app initializes
    networkRequestQueue = [...networkRequestQueue, ...persistedRequests];
    didLoadPersistedRequests = true;
    Onyx.set(ONYXKEYS.NETWORK_REQUEST_QUEUE, []);
    },
    });

    networkRequestQueue = [...networkRequestQueue, ...persistedRequests];
    We don't need to append networkRequestQueue to itself, if the network is offline, persistedRequests will have all requests we need.

    This line should be revisited,

    if (didLoadPersistedRequests || !persistedRequests) {

    didLoadPersistedRequests if it gets true once, no more requests will get appended.

  7. parasharrajat commented on Jul 28, 2021

    @parasharrajat
    Member

    We have an existing issue #2451 which solves this problem as well. PR is stalled for that. I will update you here about the PR status.

  8. thienlnam commented on Jul 29, 2021

    @thienlnam
    Contributor

    @Santhosh-Sellavel Thank you for the great proposal! The changes look good but it seems like @parasharrajat is fixing this in another PR from fixing another bug so I'm just going to have him assigned to this issue as well.

  9. 69 remaining items

  10. kidroca commented on Oct 25, 2021

    @kidroca
    Contributor

    That's one way to do it
    I think it might be simpler to move it to the next thick like so

    function Log(parameters) {
      const commandName = 'Log';
      requireParameters(['message', 'parameters', 'expensifyCashAppVersion'],
            parameters, commandName);
    
      // Note: We are forcing Log to run since it requires no authToken and should only be queued when we are offline.
      // We're scheduling the call for the next tick so that we don't re-trigger the network queue 
      return Promise.resolve()
        .then(() => Network.post(commandName, {...parameters, forceNetworkRequest: true}));
    }
    • this would add the request to the next queue

    For this to happen again we would need to add code that does a Network.post() from inside processNetworkRequestQueue(). I'm not sure why we'd expect this to happen again. This feels like a cut and dried 🐛 to me.

    I agree it's a rare thing, but it happened for Logging and can happen again, people are unaware that their change can produce such effects

    • catching a change like this in a review is not very easy - we already failed once
    • while the bug is not produced immediately but once enough logs are accumulated which again makes it hard to discover
  11. marcaaron commented on Oct 25, 2021

    @marcaaron
    Contributor

    Yeah that could work, also feels like a setTimeout() workaround. We need a note to explain why we are pushing things off to the next tick so maybe we can find a better way.

    Granted, there might never be another case where we want to avoid triggering a network request immediately with a shouldSendImmediately: false param, but feels like an approachable concept.

    • catching a change like this in a review is not very easy - we already failed once
    • while the bug is not produced immediately but once enough logs are accumulated which again makes it hard to discover

    For what it's worth, I managed to add a failing test for this so we will have some protection if someone decides it would be fun to break the behavior again.

    There's also another measure we could take here which is to just completely disallow processsNetworkRequest() to be called if it is running at all (I doubt we'd ever think about this problem again if we do this - famous last words 😅).

    let isProcessing;
    function processNetworkQueue() {
        if (isProcessing) {
            return;
        }
        // ...process
        isProcessing = false;
    }

    as long as the queue does not fail to empty in the "...process" part of the function there is no way for duplicate processing to happen and whatever is trying to happen will run on the next tick.

  12. kidroca commented on Oct 25, 2021

    @kidroca
    Contributor

    Good stuff,
    Now that I took another look I realized this is in the post method and it's simpler and clearer 👍

    if (!processImmediately) {
        return;
    }

    The isProcessing idea looks good too...

  13. mallenexpensify commented on Oct 27, 2021

    @mallenexpensify
    Contributor

    @marcaaron , tbc.... once this PR is deployed to production, there shouldn't be double messages any more? If so.. 🎉 🎉 🎉 🎉 🎉

  14. marcaaron commented on Oct 27, 2021

    @marcaaron
    Contributor

    That is the hope! We will have to wait and see.

  15. kavimuru commented on Nov 16, 2021

    @kavimuru

    @marcaaron @isagoico
    Issue was able to reproduce on the regression round for 1.1.15-1. So reopening the issue.

    Bug5330310_Sent_message.mp4

    .

  16. MelvinBot commented on Nov 19, 2021

    @MelvinBot
    Contributor

    @marcaaron Whoops! This issue is 2 days overdue. Let's get this updated quick!

  17. marcaaron commented on Nov 22, 2021

    @marcaaron
    Contributor

    Hmm there may be other causes such as messing with the cell data in this case. I think perhaps we should create a new issue with the exact reproduction steps for that. The issue with Log should be sorted. @kavimuru please open a new issue if you can and document exactly how to reproduce thanks!

  18. removed their assignment
    on Nov 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    DailyKSv2EngineeringInternalRequires API changes or must be handled by Expensify staff

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions