[$1000] Chat - App removes strikethrough formatting on copy and resend for code block

[kbecciv]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Action Performed:

1. Open the app
2. Open any report
3. Send strikethrough code block eg: Hello
4. Hover on message and click on copy to clipboard OR edit the message and copy the message
5. Send the copied message and observe that app removes strikethrough format from the code block

Expected Result:

App should keep the format of text same when we copy it to clipboard or copy from edit message and resend the message

Actual Result:

App does not keep the format of strikethrough on code block on copy to clipboard or copy from edit message and resend the message

Workaround:

Unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android / native
• Android / Chrome
• iOS / native
• iOS / Safari
• MacOS / Chrome / Safari
• MacOS / Desktop

Version Number: 1.3.41-2
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation

removes.strikethrough.on.copy.and.resend.mp4

Recording.3706.mp4

Expensify/Expensify Issue URL:
Issue reported by: GH handle - @dhanashree-sawant
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1689525870441879

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~012fc66a9c82073e51
• Upwork Job ID: 1681045679677927424
• Last Price Increase: 2023-09-11

[eh2077]

Proposal

Please re-state the problem that we are trying to solve in this issue.

App removes strikethrough formatting on copy and resend for code block.

What is the root cause of that problem?

The root cause of this issue is that we translate

~```test```~

to HTML

<del><pre>test</pre></del>

through strikethrough rule and the HTML is converted back to Markdown

~```
Hello
```
~

through htmlToMarkdownRules.

What changes do you think we should make in order to solve the problem?

To fix this issue, we can skip applying strikethrough rule if it contains pre tag, just like italic and bold rule. So, we can change this line to

replacement: (match, g1) => (g1.includes('<pre>') || this.containsNonPairTag(g1) ? match : `<del>${g1}</del>`),

By doing so, we won't apply strikethrough style for markdown input, like

~```test```~

So, copying and pasting comment will have consistent style, just like italic and bold syntax do.

What alternative solutions did you explore? (Optional)

N/A

[melvin-bot]

Triggered auto assignment to @joekaufmanexpensify (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[melvin-bot]

Bug0 Triage Checklist (Main S/O)

• This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
• This bug is not a duplicate report (check E/App issues and #expensify-bugs)
  • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
• This bug is reproducible using the reproduction steps in the OP. S/O
  • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
  • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
• This issue is filled out as thoroughly and clearly as possible
  • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
• I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

[joekaufmanexpensify]

I can reproduce this. I don't think we're really "removing" the formatting though when copying the code block. It's just that when it's pasted, there's a space between the last ` and ~. If the space is removed before you send the message you copied, we do show the strikethrough.

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~012fc66a9c82073e51

[melvin-bot]

Current assignee @joekaufmanexpensify is eligible for the External assigner, not assigning anyone new.

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @parasharrajat (External)

[parasharrajat]

I am checking the processing rules to determine the expected behaviour.

[hurtdusk]

Proposal

Please re-state the problem that we are trying to solve in this issue.

App removes strikethrough formatting on copy and resend for code block

What is the root cause of that problem?

When the "Edit Message" button is clicked, the raw text has a newline between the final ` and the second ~ which differs from the original raw message content. When a newline is introduced in this manner, the rule/pattern for strikethrough does not match. It seems that the code block is what causes the newlines to be inserted. Note that inserting newlines into any strikethrough message will cause the strikethrough to disappear.

What changes do you think we should make in order to solve the problem?

There are two options to solve this issue in my view. The first option is to make sure that when "Edit Message" is clicked, the raw text displayed is identical to the raw text that was sent originally (do not introduce newlines). The second option is to update the rule/pattern for strikethrough so that it also matches text that contains newlines.

[melvin-bot]

📣 @hurtdusk! 📣
Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork.
Please follow these steps:

1. Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
2. Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
3. Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.
   
   Format:

Contributor details
Your Expensify account email: <REPLACE EMAIL HERE>
Upwork Profile Link: <REPLACE LINK HERE>

[hurtdusk]

Contributor details
Your Expensify account email: emil05joh@gmail.com
Upwork Profile Link: https://www.upwork.com/freelancers/~012188f4918fcf5a7b

[melvin-bot]

✅ Contributor details stored successfully. Thank you for contributing to Expensify!

[mehvash-Khan]

Proposal

Please re-state the problem that we are trying to solve in this issue.

While chatting with a user, if I send ~```hello```~ to the user, copy it to clip board and paste it in the chat, the closing ~ is pasted in the next line like below:

~```
hello
```
~

What is the root cause of that problem?

Regex used in ExpensiMark.js here for converting HTML to Markdown code is adding new line i.e <br /> after closing ``` which adds a new line before closing strike through ~. This is expected, </br> should be added after closing </pre> tag, but when a new closing tag </del> appears after it does not handle it, hence not allowing it to be pasted as a striked through code block.

What changes do you think we should make in order to solve the problem?

One possible solution for this problem of adding new line after ```, can be adding an additional regex in htmlToMarkdownRules. Add an aditional condition in this regex which handles any closing tag </> which appears after </pre> and does not add </br> after </pre> if there is a closing tag following it. Please check below

{
             name: 'replacepre2',
              regex: /<\/pre>(<\/.*>)(.)/gi,
              replacement: '</pre>$1<br />$2',
}

if we want it to be specific just for strike through </del>, we can update the regex just for strike through like
regex: /<\/pre>(<\/del>)(.)/gi, and it should work as expected.

Additionally, to handle the behaviour where <br /> should be added after </pre> like it does usually, we will change the current regex replacepre to this:

{
        name: 'replacepre',
        regex: /<\/pre>(?!<\/.*>)(.)/gi,
        replacement: '</pre><br />$1',
}

This will add <br/> i.e new line after closing </pre> if there is an additional text or code after it which is not closing tag.

What alternative solutions did you explore? (Optional)

[spcheema]

Proposed solution

Please re-state the problem that we are trying to solve in this issue.

App removes strikethrough formatting on copy and resend for code block.

What is the root cause of that problem?

The root cause of this issue is that a line break is added after pre tag when parsing html to markdown.

    name: 'replacepre',
    regex: /<\/pre>(.)/gi,
    replacement: '</pre><br />$1',

pre tag is block element so any element after should placed in the new line. But strikethrough is an exceptional case.

~```some text```~

Mark down converted to html

<del><pre>some text</pre></del>

Then html further converted as below due to htmlToMarkdownRules replacepre

<del><pre>some text</pre><br /></del>

What changes do you think we should make in order to solve the problem?

Add a new rule to replace pre br del with pre del
Here is the updated code:

{ 
    name: 'replacePreBrDel',
    regex: /<\/pre><br \/><\/del>/gi,
    replacement: '</pre></del>',
},

Input markdown

~```some text```~

converted to html

<del><pre>some text</pre></del>

And <del><pre>some text</pre></del> will converted to markdown as below

~```
some text
```~

PS. This is an existing behavior in which a new line is added after/before ''' on edit.

What alternative solutions did you explore? (Optional)

This can also be solved by updating an existing regex of replacepre. But I will propose adding a new rule because, in an ideal situation, a rule should not have to worry about any other rules.

[spcheema]

POC https://www.loom.com/share/eba7a35339ca4ee9afcd84436a69c164?sid=5c8dac08-4e5e-4d64-872f-f6dc532ce4fa

[joekaufmanexpensify]

Proposals pending review

[parasharrajat]

I have stated reviewing this issue.

[joekaufmanexpensify]

Thanks! Yep, I saw. Just commented to make sure the issue doesn't go overdue today.

[parasharrajat]

Note to me: I think the root cause here is incorrect parsing of pre rule And I think that issue was stated after this PR Expensify/expensify-common#496.

[joekaufmanexpensify]

Sounds good! LMK if there's anything else I can help with as you review the proposals here.

[payal-lathidadiya]

Proposal

Please re-state the problem that we are trying to solve in this issue.

In chat - app removes strikethrough formatting on copy and resend for code block

What is the root cause of that problem?

When we've code block inside strike through ~```hello```~, it gets converted to following HTML
<del><pre>hello</pre></del>

And this HTML, gets converted back to below markup here

~```
Hello
```
~

However, above markdown doesn't get parsed as valid strikethrough section using regular expression here
regex test link

What changes do you think we should make in order to solve the problem?

To fix this, we should change strikethrough rule regex here to below;

\B~((?=\S)((~~(?!~)|[^( \t\r\f)~]|\s(?!~))+?))~\B(?![^<]*(<\/pre>|<\/code>|<\/a>))/g

Regex fix test result

Explanation:

Here, in previous regex for 3rd capturing group, 2nd alternative [^\s~] will match a single character which is not present in \s (whitespace characters i.e. [ \t\r\n\f]) followed by ~. In example given in issue, we have new line character i.e. \n before ~. So that should be excluded from that group.

What alternative solutions did you explore? (Optional)

None

[parasharrajat]

No update. Busy with critical issues.

[Skalakid]

Hey I’m Michał from Software Mansion, an expert agency, and I will help close this issue out 😄

[parasharrajat]

Go ahead. Thanks.

[Skalakid]

@eh2077 Thanks for the proposal, I really like it. I think it is a good idea to keep the behavior of strikethrough consistent with the bold and italic. Also, I can't find striking through the code block on other apps. @parasharrajat what do you think about that?

[Antasel]

@Skalakid we can find striking through in code block on discord. and italic and bold as well

[Skalakid]

@hurtdusk I'm not sure about the solution with passing raw text that was sent originally. However, updating the rule/pattern for strikethrough so that it also matches text that contains newlines, seems to be a good idea (if we want to keep striking through code blocks feature). Can you write more details about it, please?

[melvin-bot]

📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸

[parasharrajat]

I am finally free from critical issues so I will be able to check the slack thread and update you. I haven't reviewed any proposals yet because I want to check if we are setting correct parsing rules.

I opened a Slack thread but it didn't receive the needed interest so I will try to push it a little more. Currently, team is busy getting critical features out before the upcoming conference so it will take a few days before we can really move forward.

Also, I don't want to make this change at this stage to prevent unexpected bugs while the App is being used by real users. So it means we have time till the conference to decide the correct solution.

[Skalakid]

@mehvash-Khan I was testing your proposal and it fixes this bug but when I'm copying the strikethrough codeblock, it pastes with 2 new lines below it

~```
hello
```~

[Skalakid]

@spcheema I tested your solution and I cant't make it work 😅 also I think it would be better to make it not only for <del> tag but also for other formats

[spcheema]

@spcheema I tested your solution and I cant't make it work 😅 also I think it would be better to make it not only for <del> tag but also for other formats

@Skalakid Let's list all expectations and I'll make solution work. It should work as long as there isn't any change.

[Skalakid]

@payal-lathidadiya Thanks for the proposal, however I'm not sure if we want to edit strikethrough rule. On many apps such a behavior is blocked and you can't add text formatting tag when its end is the beginning of the new line (for example on GitHub/Slack).

~hello
~

[spcheema]

@spcheema I tested your solution and I cant't make it work 😅 also I think it would be better to make it not only for <del> tag but also for other formats

@Skalakid Here is a draft for testing based on the solution I proposed earlier:

Expensify/expensify-common#572

Made some changes in the proposal where only one rule is required to fix the issue.
Waiting for additional input regarding expected behaviour.

[joekaufmanexpensify]

Still pending discussions

[parasharrajat]

Expected Behaviour:

1. When ```is used inline, it should be parsed as an inline code block. It means ```hello``` will be rendered as hello.
2. If ``` is used in a multiline context, it should create code block.

 ` ` ` 
  hello
 ` ` `

will be parsed as follows

hello

3. The definition of the rule should be saved in HTML so that when the user edits the old sent message, we are able to show the original syntax used. This is necessary because now the user can use ```hello``` or hello to render the same inline code block.

4. Both MD to HTML as well as HTML to MD rules will be modified to manage this.

[parasharrajat]

Hey everyone, I have posted the expected behavior above based on the slack thread. Let me know if something looks off. If everything looks good, let's modify the OP and add this there in the expected section @joekaufmanexpensify

Please repost your proposals if old one works.

[parasharrajat]

Please hold your proposals. I think there are a couple of issues with the nesting of rules that need a proper discussion and design.

I have opened another thread to see what others think about it. https://expensify.slack.com/archives/C02NK2DQWUX/p1694008860474759 https://expensify.slack.com/archives/C02NK2DQWUX/p1694009190325299
https://expensify.slack.com/archives/C01GTK53T8Q/p1694011229493309

[joekaufmanexpensify]

Sounds good!

[joekaufmanexpensify]

@parasharrajat is there anyone specific you're looking for input from on those threads? If so, I'm happy to just tag them so we can try to push this forward.

[parasharrajat]

I don't want to forcefully pull anyone into the discussion. I hope that the engineering team will be interested in participating. You know the team better than me so please tag the persons you want to.

[joekaufmanexpensify]

I'm sure people will be interested! Just concerned the thread might have gotten buried in the room. I see there's been more input, but I will bump it now.

[melvin-bot]

📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸

[thienlnam]

Closing in favor of #27208
Discussion here: https://expensify.slack.com/archives/C01GTK53T8Q/p1694011229493309

[dhanashree-sawant]

Hi @thienlnam, you have written the same issue number in your message, maybe you were marking any other issue?

[thienlnam]

Updated, thanks