Skip to content

[HOLD for payment 2023-10-02] [$1000] Chat - Mention is not displayed in message with full 15000 letters #21817

Closed

Description

@kbecciv

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!


Action Performed:

  1. Open the app
  2. Open any report
  3. Write message with 15000 letters (max limit) and include mention in it in middle or in end
  4. Observe that app does not recognize mention in the message

Expected Result:

App should recognize mention in message with 15000 letters (max limit)

Actual Result:

App does not recognize mention in message with 15000 letters (max limit)

Workaround:

Unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

  • Android / native
  • Android / Chrome
  • iOS / native
  • iOS / Safari
  • MacOS / Chrome / Safari
  • MacOS / Desktop

Version Number: 1.3.33-3
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation

mention.not.working.in.message.with.max.length.2.mp4
Recording.3294.mp4

Expensify/Expensify Issue URL:
Issue reported by: @dhanashree-sawant
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1687872696950149

View all open jobs on GitHub

Upwork Automation - Do Not Edit
  • Upwork Job URL: https://www.upwork.com/jobs/~015b3ce1ef6e882fd1
  • Upwork Job ID: 1674255187768983552
  • Last Price Increase: 2023-07-06

Activity

  1. added
    BugSomething is broken. Auto assigns a BugZero manager.
    on Jun 28, 2023
  2. melvin-bot commented on Jun 28, 2023

    @melvin-bot

    Triggered auto assignment to @sophiepintoraetz (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

  3. melvin-bot commented on Jun 28, 2023

    @melvin-bot

    Bug0 Triage Checklist (Main S/O)

    • This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
    • This bug is not a duplicate report (check E/App issues and #expensify-bugs)
      • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
    • This bug is reproducible using the reproduction steps in the OP. S/O
      • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
      • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
    • This issue is filled out as thoroughly and clearly as possible
      • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
    • I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync
  4. samh-nl commented on Jun 28, 2023

    @samh-nl
    Contributor

    Proposal

    Please re-state the problem that we are trying to solve in this issue.

    Mentions are not displayed in comments that approach the 15000 character limit.

    What is the root cause of that problem?

    MAX_COMMENT_LENGTH is higher than MAX_MARKUP_LENGTH, leading to long messages not being displayed with markdown.

    App/src/CONST.js

    Line 1233 in ecfcd83

    MAX_MARKUP_LENGTH: 10000,

    What changes do you think we should make in order to solve the problem?

    To correctly display markdown in all messages: MAX_MARKUP_LENGTH should be equal to MAX_COMMENT_LENGTH.

    However, this leaves an edge case in the composer, where it is possible to type messages that exceed MAX_COMMENT_LENGTH. The message in the composer is parsed prior to determining its length, as this is how the BE counts messages also.

    App/src/libs/ReportUtils.js

    Lines 2277 to 2281 in 67bac9d

    function getCommentLength(textComment) {
    return getParsedComment(textComment)
    .replace(/[^ -~]/g, '\\u????')
    .trim().length;
    }

    App/src/libs/ReportUtils.js

    Lines 1289 to 1299 in 67bac9d

    /**
    * For comments shorter than 10k chars, convert the comment from MD into HTML because that's how it is stored in the database
    * For longer comments, skip parsing, but still escape the text, and display plaintext for performance reasons. It takes over 40s to parse a 100k long string!!
    *
    * @param {String} text
    * @returns {String}
    */
    function getParsedComment(text) {
    const parser = new ExpensiMark();
    return text.length < CONST.MAX_MARKUP_LENGTH ? parser.replace(text) : _.escape(text);
    }

    As you remove characters from a long message, the character count will decrease but then suddenly jump as markdown parsing will be applied. Please see the video below for a demonstration.

    Video

    chrome_RO2IrooXyk.mp4

    In conclusion, we can consider these solutions:

    Option 1. Keep the current counting algorithm and increase MAX_MARKUP_LENGTH to a value higher than MAX_COMMENT_LENGTH.
    This ensures the entire message is parsed, as well as prevents the message length jumping behavior in the composer near MAX_COMMENT_LENGTH.

    Option 2. Make MAX_MARKUP_LENGTH equal to MAX_COMMENT_LENGTH, then:

    • getCommentLength must be changed to count the unprocessed message (i.e. how it's typed by the user). This counting change should also be applied to the BE, which may make it more complex as the BE does not currently see the plaintext message and secondly due to storage limit considerations and the need to validate this. After all, post-markdown processing is how the message is actually stored in the BE.
    • text.length < CONST.MAX_MARKUP_LENGTH should be made inclusive, i.e. <= instead of <

    Option 3. Optimize markdown processing speed and remove MAX_MARKUP_LENGTH altogether (or drastically increase)
    Slack does not have any responsiveness issues when typing long messages (tried 375k+) in their composer and only limit the length of pasting long texts. I think there is plenty room for optimization and the only reason of existence (to my knowledge) for MAX_MARKUP_LENGTH is to prevent performance degradation.

    Marked.js handles complex 100k markdown texts in the ~50ms range. CommonMark requires ~20ms. Expensify's implementation however requires tens of seconds (freezing the tab) due to its heavy reliance on regular expressions. It's a simple, but poorly performant implementation. Arguably this is the true root cause of the problem.

    What alternative solutions did you explore? (Optional)

    N/A

    EDIT: Revised proposal.

  5. melvin-bot commented on Jun 28, 2023

    @melvin-bot

    📣 @samh-nl! 📣
    Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork.
    Please follow these steps:

    1. Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
    2. Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
    3. Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.
      Screen Shot 2022-11-16 at 4 42 54 PM
      Format:
    Contributor details
    Your Expensify account email: <REPLACE EMAIL HERE>
    Upwork Profile Link: <REPLACE LINK HERE>
    
  6. yh-0218 commented on Jun 28, 2023

    @yh-0218
    Contributor

    Proposal

    Please re-state the problem that we are trying to solve in this issue.

    App should recognize mention in message with 15000 letters (max limit)

    What is the root cause of that problem?

    MAX_MARKUP_LENGTH and MAX_COMMENT_LENGTH are different.

    What changes do you think we should make in order to solve the problem?

    MAX_COMMENT_LENGTH to 10,000 or MAX_MARKUP_LENGTH to 15,000

    What alternative solutions did you explore? (Optional)

  7. sophiepintoraetz commented on Jun 29, 2023

    @sophiepintoraetz
    Contributor

    2023-06-29_15-13-26 (1)

  8. added
    ExternalAdded to denote the issue can be worked on by a contributor
    on Jun 29, 2023
  9. changed the title [-]Chat - Mention is not displayed in message with full 15000 letters[/-] [+][$1000] Chat - Mention is not displayed in message with full 15000 letters[/+] on Jun 29, 2023
  10. melvin-bot commented on Jun 29, 2023

    @melvin-bot
  11. added
    Help WantedApply this label when an issue is open to proposals by contributors
    on Jun 29, 2023
  12. melvin-bot commented on Jun 29, 2023

    @melvin-bot

    Current assignee @sophiepintoraetz is eligible for the External assigner, not assigning anyone new.

  13. 119 remaining items

  14. sobitneupane commented on Oct 9, 2023

    @sobitneupane
    Contributor

    BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

    This issue is resulted by multiple PRs and was kind of expected result back then.

  15. sobitneupane commented on Oct 9, 2023

    @sobitneupane
    Contributor

    Regression Test Proposal

    1. Open any report
    2. Type a message that ends with a user mention and where the message length exceeds the 10,000 limit
    3. Verify that an error indicator appears
    4. Reduce the message length until the error indicator disappears, but ensure the user mention is still present
    5. Post the message
    6. Verify that the user mention is correctly processed (i.e. is not plaintext)

    Do we agree 👍 or 👎

  16. sobitneupane commented on Oct 9, 2023

    @sobitneupane
    Contributor

    #21817 (comment)

    Requested payment on newDot.

  17. JmillsExpensify commented on Oct 9, 2023

    @JmillsExpensify
    Contributor

    $1,000 payment approved for @sobitneupane based on BZ summary.

  18. JmillsExpensify commented on Oct 9, 2023

    @JmillsExpensify
    Contributor

    Re-opening this issue since we still need the regression tested added to TestRail.

  19. neil-marcellini commented on Oct 12, 2023

    @neil-marcellini
    Contributor

    Not over due

  20. melvin-bot commented on Oct 16, 2023

    @melvin-bot

    @sobitneupane, @neil-marcellini, @NickTooker, @sophiepintoraetz, @samh-nl Uh oh! This issue is overdue by 2 days. Don't forget to update your issues!

  21. removed their assignment
    on Oct 16, 2023
  22. sophiepintoraetz commented on Oct 18, 2023

    @sophiepintoraetz
    Contributor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Awaiting PaymentAuto-added when associated PR is deployed to productionBugSomething is broken. Auto assigns a BugZero manager.DailyKSv2Engineering

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions