[HOLD for payment 2023-10-13] [$500] Chat - Strikethrough link isn't parsed correctly on link with URL that contains tilde

[izarutskaya]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Action Performed:

1. Open a chat
2. Send this message "~ [Example Link]([https://example.com?example= ~ ex)~ ](https://example.com/?example= ~ ex) ~)"
3. Examine how the text parsed

Expected Result:

The tilde should be parsed as styling character, and the strikethrough should be applied (like underscore and italicized link that behave like that, so this could be categorized as inconsistency issue as well)

Actual Result:

The tilde isn't parsed and the message isn't styled as expected

Workaround:

Unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android / native
• Android / Chrome
• iOS / native
• iOS / Safari
• MacOS / Chrome / Safari
• MacOS / Desktop

Version Number: v1.3.69-0

Reproducible in staging?: Y

Reproducible in production?: Y

If this was caught during regression testing, add the test name, ID and link from TestRail:

Email or phone of affected tester (no customers):

Logs: https://stackoverflow.com/c/expensify/questions/4856

Notes/Photos/Videos: Any additional supporting documentation

Strikethrough.Link.mp4

Recording.1519.mp4

Expensify/Expensify Issue URL:

Issue reported by: @kerupuksambel

Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1694320083877809

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~0193c33b549a6d01ac
• Upwork Job ID: 1701921122476535808
• Last Price Increase: 2023-09-20
• Automatic offers:
• ZhenjaHorbach | Contributor | 26877283
• kerupuksambel | Reporter | 26877284

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~0193c33b549a6d01ac

[melvin-bot]

Triggered auto assignment to @greg-schroeder (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[melvin-bot]

Bug0 Triage Checklist (Main S/O)

• This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
• This bug is not a duplicate report (check E/App issues and #expensify-bugs)
  • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
• This bug is reproducible using the reproduction steps in the OP. S/O
  • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
  • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
• This issue is filled out as thoroughly and clearly as possible
  • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
• I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

[melvin-bot]

Triggered auto assignment to @sophiepintoraetz (External), see https://stackoverflow.com/c/expensify/questions/8582 for more details.

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @robertKozik (External)

[sahilbhosale63]

Hi @izarutskaya
I just now saw this issue in Upwork job posting and I wanted to contribute to this issue. I downloaded the source and ran the web app on local but I'm facing the "You appear to be offline" issue frequently while I am using the app. Due this I'm not able to debug the issue correctly. I have stable internet but still I'm facing this issue. Can you please guide me here?

[melvin-bot]

📣 @sahilbhosale63! 📣
Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork.
Please follow these steps:

1. Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
2. Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
3. Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.
   
   Format:

Contributor details
Your Expensify account email: <REPLACE EMAIL HERE>
Upwork Profile Link: <REPLACE LINK HERE>

[ZhenjaHorbach]

Proposal

Please re-state the problem that we are trying to solve in this issue

After sending a complex message with ~ ~, the text is not crossed out

What is the root cause of that problem?

The main problem is the regular expression that is used in our common library

https://github.com/Expensify/expensify-common/blob/35bff866a8d345b460ea6256f0a0f0a8a7f81086/lib/ExpensiMark.js#L211-L215

Which incorrectly processes links containing ~

For example:
Match for [Example Link] (https://example.com/?example=**~**ex) - ex" target="_blank" rel="noreferrer noopener">Example Link

Match for [Example Link] (https://example.com/?example=**_**ex) - < a hre f= "https://example.com/?example=~ ex" target= "_blank" rel="noreferrer noopener"> Example Link </ a >

As a result here we get incorrect result

https://github.com/Expensify/expensify-common/blob/35bff866a8d345b460ea6256f0a0f0a8a7f81086/lib/ExpensiMark.js#L214

What changes do you think we should make in order to solve the problem?

I'm not sure if it's possible to make changes to the common library, but it would be nice to make a few changes to the regular expression
So that it would work by analogy with italic regular

https://github.com/Expensify/expensify-common/blob/35bff866a8d345b460ea6256f0a0f0a8a7f81086/lib/ExpensiMark.js#L142-L165

What alternative solutions did you explore? (Optional)

Also we have a special function that allows us to apply strikethrough
But this function doesn't do anything with the text assuming that the component already has a <del > wrapper
So we can add a condition that will add <del/>

Like

function applyStrikethrough(html) {
    if (html.match(/^(~).*\1$|^~$/)) {
        return `<del>${html}</del>`;
    }
    return html;
}

Or we can add this condition in getParsedComment

App/src/libs/ReportUtils.js

Lines 1771 to 1774 in 86d37d6

	function getParsedComment(text) {
	const parser = new ExpensiMark();
	return text.length < CONST.MAX_MARKUP_LENGTH ? parser.replace(text) : _.escape(text);
	}

[astrohunter62]

Proposal

Please re-state the problem that we are trying to solve in this issue.

The strikethrough link does not parse correctly when the URL contains a tilde.

What is the root cause of that problem?

In expensify-common library, the link with URL that contains tilde is not handled correctly.

What changes do you think we should make in order to solve the problem?

If it is possible to modify the module, add a regular expression to the modifyTextForUrlLinks function in `node_modules/expensify-common/lib/ExpensiMark.js``.
For example:

        ...
        return replacedText.replace(/~(.*?)~|~/g, '<del>$1</del>');
       ...

Alternatively, we can add a regular expression to getParsedComment() in src/lib/ReportUtils.js.
For example:

function applyStrikethrough(html) {
    return html.replace(/~(.*?)~|~/g, '<del>$1</del>');
}
function getParsedComment(text) {
    const parser = new ExpensiMark();
    return text.length < CONST.MAX_MARKUP_LENGTH ? applyStrikethrough(parser.replace(text)) : _.escape(text);
}

Result:

result.webm

[Antasel]

Proposal

Please re-state the problem that we are trying to solve in this issue

The current strikethrough markdown is not applied if url contains tilde.
This kind of issue also happens with bold markdown.

What is the root cause of that problem?

Current regex patterns of strikethrough and bold rule are not covering this issue case.

What changes do you think we should make in order to solve the problem?

We need to fix regex pattern on strikethrough, also bold rules.

As following:

            {
                name: 'bold',
                regex: /\B\*((?![\s\*])[\s\S]*?[^\s\*])\*\B(?![^<]*(<\/pre>|<\/code>|<\/a>))/g,
                replacement: (match, g1) => (g1.includes('<pre>') || this.containsNonPairTag(g1) ? match : `<strong>${g1}</strong>`),
            },
            {
                name: 'strikethrough',
                regex: /\B~((?![\s~])[\s\S]*?[^\s~])~\B(?![^<]*(<\/pre>|<\/code>|<\/a>))/g,
                replacement: (match, g1) => (this.containsNonPairTag(g1) ? match : `<del>${g1}</del>`),
            },

and we can add test cases.

testString = '~test.com/test=~123~';
resultString = '<del><a href=\"https://test.com/test=~123\" target=\"_blank\" rel=\"noreferrer noopener\">test.com/test=~123</a></del>'

testString = '*test.com/test=*123*';
resultString = '<strong><a href=\"https://test.com/test=*123\" target=\"_blank\" rel=\"noreferrer noopener\">test.com/test=*123</a></strong>'

The solution does not make other test cases failed.

[greg-schroeder]

awaiting proposal review

[melvin-bot]

📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸

[robertKozik]

Sorry for keeping you waiting. Going to review the proposals now

[robertKozik]

Hi all @Antasel @astrohunter62 @ZhenjaHorbach! Thank you all for your proposals. Yes, we can do the PR to expensify-common I believe. All of you pointed out correct root cause and propose changing the regex in expensify-common. That's why I believe we should select the first contributor to propose this - @ZhenjaHorbach.

Selected Proposals: #27329 (comment)
Contributor: @ZhenjaHorbach

[greg-schroeder]

@robertKozik can you make sure to use the proper formatting to assign an internal engineer to confirm the proposal selection? like the 🎀 👀 🎀 C+ reviewed! thing

[Beamanator]

PR merged today!

[ZhenjaHorbach]

@Beamanator
Hello )
Sorry for the stupid question)
When we merged changes into the common library
Do we need to make any changes to the main project?)

[Beamanator]

Not a stupid question! But yes, you can see in package.json of this project that our expensify-common package points to the latest commit in the expensify-common repo, and since we just merged an update there, you'll have to also update package.json & i believe package-lock.json as well - you can see recent PRs that update that line to see examples 👍

[ZhenjaHorbach]

@Beamanator
Now everything has become clear)
Thanks for the help )

[melvin-bot]

Based on my calculations, the pull request did not get merged within 3 working days of assignment. Please, check out my computations here:

• when @ZhenjaHorbach got assigned: 2023-09-26 06:08:42 Z
• when the PR got merged: 2023-10-04 11:53:36 UTC
• days elapsed: 6

On to the next one 🚀

[melvin-bot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.3.78-4 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• Strikethrough link is not parsed correctly on link with URL that contains tilde #28616

If no regressions arise, payment will be issued on 2023-10-13. 🎊

After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

• External issue reporter
• Contributor that fixed the issue
• Contributor+ that helped on the issue and/or PR

For reference, here are some details about the assignees on this issue:

• @robertKozik does not require payment (Contractor)
• @ZhenjaHorbach requires payment offer (Contributor)
• @kerupuksambel requires payment offer (Reporter)

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

• Merged PR within 3 business days of assignment - 50% bonus
• Merged PR more than 9 business days after assignment - 50% penalty

[melvin-bot]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

• [@robertKozik] The PR that introduced the bug has been identified. Link to the PR:
• [@robertKozik] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
• [@robertKozik] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
• [@robertKozik] Determine if we should create a regression test for this bug.
• [@robertKozik] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
• [@greg-schroeder] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

[greg-schroeder]

This is ready for payment tomorrow unless any regressions pop up

[robertKozik]

• [@robertKozik] The PR that introduced the bug has been identified. Link to the PR: It's not a regression from the PR, it's new edge-case which was not covered yet
• [@robertKozik] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment: N/A
• [@robertKozik] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion: N/A
• [@robertKozik] Determine if we should create a regression test for this bug. I think we don't need it, we have new unit tests for such a cases inside expensify-common repo
• [@robertKozik] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again. N/A

[melvin-bot]

@Beamanator, @greg-schroeder, @robertKozik, @ZhenjaHorbach Uh oh! This issue is overdue by 2 days. Don't forget to update your issues!

[Beamanator]

I believe this is ready for payment 👍

[greg-schroeder]

Yes, sorry, I was sick on Friday

[greg-schroeder]

Issue Participants:

Issue reported by: @kerupuksambel
Contributor: @ZhenjaHorbach
C+: @robertKozik

Was this issue merged in time to be eligible for the speed bonus? No (context)
Were their any regressions? No

Payment summary:

Reporter: $50
Contributor: $500
C+: n/a

[greg-schroeder]

Payments made, closing.