[HOLD for payment 2023-10-16] [$500] Chat - App removes space from start for text below header text

[kbecciv]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Action Performed:

1. Open the app
2. Open any report
3. Send message with normal text in first line and some space before text in next line
4. Copy that text or edit and observe that space is preserved in second line which is correct
5. Now send message with header text (# in front) in first line and some space before text in next line
6. Copy that text or edit and observe that it removes the space before second line text

Expected Result:

App should preserve space added before text in second or any other line except first line

Actual Result:

App does not preserve space added before text in second or any other line if first line has header text (text with # in beginning)

Workaround:

Unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android / native
• Android / Chrome
• iOS / native
• iOS / Safari
• MacOS / Chrome / Safari
• MacOS / Desktop

Version Number: 1.3.70.2
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation

space.before.text.not.preserved.below.header.text.mp4

Recording.4465.mp4

Expensify/Expensify Issue URL:
Issue reported by: @dhanashree-sawant
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1694453805196449

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~01a29ca52e1304c30b
• Upwork Job ID: 1702326265887232000
• Last Price Increase: 2023-09-14
• Automatic offers:
• s-alves10 | Contributor | 26744986
• dhanashree-sawant | Reporter | 26744992

Issue Owner

Current Issue Owner: @anmurali

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~01a29ca52e1304c30b

[melvin-bot]

Triggered auto assignment to @anmurali (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[melvin-bot]

Bug0 Triage Checklist (Main S/O)

• This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
• This bug is not a duplicate report (check E/App issues and #expensify-bugs)
  • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
• This bug is reproducible using the reproduction steps in the OP. S/O
  • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
  • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
• This issue is filled out as thoroughly and clearly as possible
  • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
• I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

[melvin-bot]

Triggered auto assignment to @peterdbarkerUK (External), see https://stackoverflow.com/c/expensify/questions/8582 for more details.

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @parasharrajat (External)

[s-alves10]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Copying or editing the message removes spaces at the beginning of the line if the previous line is a header text. If the next line is a header text, copying or editing removes trailing spaces

What is the root cause of that problem?

We have html-to-markdown rule regarding heading here

        regex: /[^\S\r\n]*<(h1)(?:"[^"]*"|'[^']*'|[^'">])*>(.*?)<\/\1>(?![^<]*(<\/pre>|<\/code>))[^\S\r\n]*/gi,

As you can see, we match space characters(and \r\n) before and after the heading tag. The result is shown below

This is the root cause

What changes do you think we should make in order to solve the problem?

We need to match only the <h1>xxx</h1> part of the string
Removing [^\S\r\n]* from the start and end will preserve all space characters and newline characters before and after the heading string. So the new regex would be

        regex: /<(h1)(?:"[^"]*"|'[^']*'|[^'">])*>(.*?)<\/\1>(?![^<]*(<\/pre>|<\/code>))/gi,

There are two test errors by this change

test('Test html to heading1 markdown when h1 tag is in the beginning of the line', () => {
    const testString = '<h1>heading1</h1> in the beginning of the line';
    const resultString = '# heading1\n'
    + 'in the beginning of the line';
    expect(parser.htmlToMarkdown(testString)).toBe(resultString);
});

    - in the beginning of the line
    +  in the beginning of the line

test('Test html to heading1 markdown when h1 tags are in the middle of the line', () => {
    const testString = 'this line has a <h1>heading1</h1> in the middle of the line';
    const resultString = 'this line has a\n'
    + '# heading1\n'
    + 'in the middle of the line';
    expect(parser.htmlToMarkdown(testString)).toBe(resultString);
});

    - this line has a
    + this line has a 
      # heading1
    - in the middle of the line
    +  in the middle of the line

This is something expected in this issue.

Note: In case we don't want to preserve before the heading, we should keep [^\S\r\n]* at the beginning, but I don't think we need to discard them.

Result

27445.mp4

What alternative solutions did you explore? (Optional)

[jaylalakiya]

Proposal

Please re-state the problem that we are trying to solve in this issue.

App removes spaces before line where previous line is header text when pasting the copied text in chat. This problem is not only with selected platforms in issue description, but instead it persists on all the platforms.

What is the root cause of that problem?

When the html is converted to markdown it removes trailing spaces after heading tag even if there is text after trailing spaces which causes this behaviour of removing leading spaces from next line

What changes do you think we should make in order to solve the problem?

We should modify regex to not remove trailing spaces because trailing spaces are trimmed somewhere else.
Current regex: /[^\S\r\n]*<(h1)(?:"[^"]*"|'[^']*'|[^'">])*>(.*?)<\/\1>(?![^<]*(<\/pre>|<\/code>))[^\S\r\n]*/gi
We should remove trailing spaces regex [^\S\r\n]*

This is not a breaking change as it only removes trailing spaces and not leading spaces (see video attached below). Trailing spaces are trimmed here in case there is no text after spaces.

Result:
https://github.com/Expensify/App/assets/63226109/0982d02f-58f3-4213-a5ba-7b08393d1891

What alternative solutions did you explore? (Optional)

[parasharrajat]

@s-alves10 Please re-state the problem that we are trying to solve in this issue. We are expecting a more generalized summary of your understanding of the problem. Repeating the title does not satisfy this.

Please fix it and do the same in the future.

[s-alves10]

@parasharrajat

Issue description updated

[jaylalakiya]

@parasharrajat Please can you review this
comment

[melvin-bot]

@anmurali, @peterdbarkerUK, @parasharrajat Whoops! This issue is 2 days overdue. Let's get this updated quick!

[parasharrajat]

@jaylalakiya Please close that PR. It is not authorized. Did you get a chance to read the contribution guidelines?

[parasharrajat]

@s-alves10 Your proposal looks fine to me. But Can you tell me if it does not break Expensify/expensify-common#531?

Also, I think we should trip whitespaces before the heading. it does not make sense to have spaces before heading.

[s-alves10]

@parasharrajat

But Can you tell me if it does not break Expensify/expensify-common#531?

Yeah. Tests include them but my solution passed all of them

I think we should trip whitespaces before the heading. it does not make sense to have spaces before heading.

We can keep [^\S\r\n]* before

[parasharrajat]

Ok. Thanks for clarifying. I think we can move ahead with @s-alves10's proposal as they proposed first and it solves the issue.

We will continue stripping the preceding spaces in the heading rule. We will also adjust the tests to correctly set an expectation from the rule. @s-alves10 We will have to fix any inconsistencies found during PR testing from our changes. I expect that there will be more changes to satisfy all conditions.

🎀 👀 🎀 C+ reviewed

[melvin-bot]

Triggered auto assignment to @marcaaron, see https://stackoverflow.com/c/expensify/questions/7972 for more details.

[melvin-bot]

Based on my calculations, the pull request did not get merged within 3 working days of assignment. Please, check out my computations here:

• when @s-alves10 got assigned: 2023-09-19 03:20:52 Z
• when the PR got merged: 2023-10-05 22:09:56 UTC
• days elapsed: 12

On to the next one 🚀

[s-alves10]

PRs were merged without any changes.

[melvin-bot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.3.79-5 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• fix: keep trailing spaces in heading #28674

If no regressions arise, payment will be issued on 2023-10-16. 🎊

After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

• External issue reporter
• Contributor that fixed the issue
• Contributor+ that helped on the issue and/or PR

For reference, here are some details about the assignees on this issue:

• @parasharrajat does not require payment (Eligible for Manual Requests)
• @s-alves10 requires payment offer (Contributor)
• @dhanashree-sawant requires payment offer (Reporter)

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

• Merged PR within 3 business days of assignment - 50% bonus
• Merged PR more than 9 business days after assignment - 50% penalty

[melvin-bot]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

• [@parasharrajat] The PR that introduced the bug has been identified. Link to the PR:
• [@parasharrajat] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
• [@parasharrajat] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
• [@parasharrajat] Determine if we should create a regression test for this bug.
• [@parasharrajat] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
• [@anmurali] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

[melvin-bot]

@anmurali, @parasharrajat, @marcaaron, @s-alves10 Whoops! This issue is 2 days overdue. Let's get this updated quick!

[parasharrajat]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

• [@parasharrajat] The PR that introduced the bug has been identified. Link to the PR: Avoid removing break before and after heading expensify-common#531
• [@parasharrajat] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment: https://github.com/Expensify/expensify-common/pull/531/files#r1365793978
• [@parasharrajat] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion: This could have been caught during testing.
• [@parasharrajat] Determine if we should create a regression test for this bug. yes
• [@parasharrajat] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.

Regression Test Steps

1. Open any chat and send a message with header text(# in front) in the first line and some text starting with a few spaces in the next line.
2. Copy that text or switch the message to edit mode
3. Verify that the spaces before the second line aren't removed.

Do you agree 👍 or 👎 ?

[melvin-bot]

@anmurali, @parasharrajat, @marcaaron, @s-alves10 Uh oh! This issue is overdue by 2 days. Don't forget to update your issues!

[anmurali]

• @parasharrajat $500 (Eligible for Manual Requests)
• @s-alves10 $500 offer (Contributor)
• @dhanashree-sawant $50 offer (Reporter)

[melvin-bot]

@anmurali, @parasharrajat, @marcaaron, @s-alves10 Huh... This is 4 days overdue. Who can take care of this?

[parasharrajat]

We can close this issue. I am tracking the payment.

[parasharrajat]

Payment requested as per #27445 (comment)

[JmillsExpensify]

$500 approved for @parasharrajat