[HOLD for payment 2023-05-05] [$1000] Invalid urls in chat are parsed as valid, redirecting to the app again.

[kavimuru]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Action Performed:

1. Open any chat
2. Enter an invalid url with hyphen in the end(before TLD)
3. Now open that url

Expected Results:

should not be parsed as url, and if parsed, then should go to that invalid url ending up with an error from search engine on screen

Actual results:

parsed as url, and then on opening that, it redirects to the app again

Workaround:

Can the user still use Expensify without this being fixed? Have you informed them of the workaround?

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android / native
• Android / Chrome
• iOS / native
• iOS / Safari
• MacOS / Chrome / Safari
• MacOS / Desktop

Version Number: 1.3.1
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation

Screenrecorder-2023-04-16-20-07-14-845.mp4

Recording.265.mp4

Expensify/Expensify Issue URL:
Issue reported by: @chiragxarora
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1681656328792209

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~019d3fae18911c3787
• Upwork Job ID: 1648700378826989568
• Last Price Increase: 2023-04-19

[MelvinBot]

Triggered auto assignment to @kadiealexander (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[kadiealexander]

Reassigning as I'm heading OOO for a week.

[MelvinBot]

Triggered auto assignment to @slafortune (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[MelvinBot]

Bug0 Triage Checklist (Main S/O)

• This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
• This bug is not a duplicate report (check E/App issues and #expensify-bugs)
  • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
• This bug is reproducible using the reproduction steps in the OP. S/O
  • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
  • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
• This issue is filled out as thoroughly and clearly as possible
  • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
• I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

[slafortune]

Looks good!

[MelvinBot]

Job added to Upwork: https://www.upwork.com/jobs/~019d3fae18911c3787

[MelvinBot]

Current assignee @slafortune is eligible for the External assigner, not assigning anyone new.

[MelvinBot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @mananjadhav (External)

[MelvinBot]

Triggered auto assignment to @jasperhuangg (External), see https://stackoverflow.com/c/expensify/questions/7972 for more details.

[jjcoffee]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Invalid URLs are still parsed by the autolinker as if they are URLs, which results in the weird redirecting behaviour.

What is the root cause of that problem?

The ExpensiMark autolinker is not validating URLs in the same way as the BE, specifically it is accepting URLs that start or end with a - as valid. You can see if you look closely that the link initially shows as underlined and then the underline goes away (once the BE request finishes and updates the FE). Basically the BE is stripping the URL from the href, leaving it empty (which then causes the odd redirect behaviour).

What changes do you think we should make in order to solve the problem?

We need to correct the autolinker's regex so that URLs that start or end with a - are not autolinked, as they are invalid.

We need to update the URL_WEBSITE_REGEX here:

https://github.com/Expensify/expensify-common/blob/e93e1eb448ad6bdbde911fd6239f70d5e749635e/lib/Url.js#L3

(https?:\\/\\/)?((?:www\\.)?[a-z0-9][-a-z0-9]+?[a-z0-9]\\.)+(?:${TLD_REGEX})(?:\\:\\d{2,4}|\\b|(?=_))

This results in test-.com being displayed as normal text, whereas for -test.com, only the valid domain part is rendered as a link (this is consistent with how we deal with ~test.com, for example).

What alternative solutions did you explore? (Optional)

It might be useful for a retrospective fix here too so that previously sent links that got caught up in this don't render incorrectly. This could be done either in the BE, or we can choose to not render empty links on the FE (I think this is a bit out of scope for this bug, though!).

[jasperhuangg]

@jjcoffee The URL abc-.com is technically still a valid URL, so I think we still want it to be parsed by the regex. If clicked we just want it to perform like a regular URL and open in a different tab.

Your solution prevents the URL from being highlighted, which isn't what we want.

[jjcoffee]

@jasperhuangg The abc- label in the abc-.com URL represents an invalid domain name (they cannot start or end with a hyphen), which is why (I think!) the BE is stripping it out as it's impossible for the URL to go anywhere.

If you still think we want to accept it as a valid URL, I think BE would need to be changed.

[ahmedGaber93]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Invalid urls in chat redirecting to the app again.

What is the root cause of that problem?

After network debuging, it is a backend issue

// in `api?command=AddComment` api we send correct <a> with href attribute.
reportComment: <a href="https://abc-.com" target="_blank" rel="noreferrer noopener">https://abc-.com</a>

// but in `api?command=OpenReport` is received without href attribute
`"html": "<a target="_blank" rel="noreferrer noopener">https://abc-.com</a>"`

What changes do you think we should make in order to solve the problem?

The fix should be done from backend side.

What alternative solutions did you explore? (Optional)

[MelvinBot]

@mananjadhav, @slafortune, @jasperhuangg Whoops! This issue is 2 days overdue. Let's get this updated quick!

[jasperhuangg]

ah @jjcoffee I see what you mean, thanks I didn't know! If that's the case I'm happy to move forward with your solution.

[jjcoffee]

@slafortune Are you able to hire me on the Upwork job ready for future payment, when you get a chance?

[chiragxarora]

I also haven't been hired for the bug report yet and sadly I don't have much connects to apply. Could you please send me an offer instead?

My profile link: https://www.upwork.com/freelancers/~014a1b566a7d8ee0c5?s=1110580755107926016

[mananjadhav]

@jjcoffee I thought we're covering both the cases. Leading and trailing hyphens. It looks like leading hyphen is still being rendered as a link.

[jjcoffee]

@mananjadhav This is expected (only the URL part is highlighted, without the hyphen), as stated in my proposal:

This results in test-.com being displayed as normal text, whereas for -test.com, only the valid domain part is rendered as a link (this is consistent with how we deal with ~test.com, for example).

[jjcoffee]

The other PR with the newer version of expensify-common that also has my fix has now been deployed to production. If no regressions arise, payment should be issued on 2023-05-05. cc @slafortune

[chiragxarora]

Hi @slafortune could you pls update this issue?

[mountiny]

@jjcoffee @mananjadhav can you fill out the bug zero checklist here (from some other issue where it got posted)

And also break down what shoul dbe the rewards here.

[mananjadhav]

@mountiny @jasperhuangg I unable to pinpoint the offending PR as it looks like it didn't work since a long time and not due to the recent changes.

Here's the regression test proposal.

1. Open any chat
2. Enter an invalid URL ending with a hyphen (e.g., abc-.com) in the message and send
3. Verify that the message is not hyperlinked
4. Ensure the message is successfully sent to the API and then still the message shouldn't be hyperlink
5. Now enter another invalid URL starting with a hyphen e.g -abc.com in the message and send
6. Verify that excluding the hyphen, the rest of the text is highlighted as hyperlink
7. Again wait for the message to be sent via API and verify the behavior as point 6.

Do we agree 👍 or 👎 ?

What do you folks think @jasperhuangg @mountiny @jjcoffee @slafortune ?

[jjcoffee]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

• [@mananjadhav] The PR that introduced the bug has been identified. Link to the PR:
• [@mananjadhav] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
• [@mananjadhav] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
• [@mananjadhav] Determine if we should create a regression test for this bug.
• [@mananjadhav] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
• [@slafortune] Link the GH issue for creating/updating the regression test once above steps have been agreed upon: https://github.com/Expensify/Expensify/issues/281920

[jjcoffee]

50% timeliness bonus is due as this was assigned 24 April, PR (in expensify-common) merged April 25. So I make it $1,500 for contributors (@jjcoffee @mananjadhav) and $250 for the reporting bonus to @chiragxarora. I have applied to the Upwork job @slafortune.

[chiragxarora]

There are 2 contributors for this issue?

[jjcoffee]

@chiragxarora I meant @mananjadhav is the C+

[slafortune]

@mananjadhav - can you check off any pertaining steps in the BZ checklist above?
@chiragxarora @mananjadhav @jjcoffee contracts are sent 👍

[jjcoffee]

@slafortune Thanks Stevie - contract accepted!

[mananjadhav]

@slafortune Can you please refer this comment for the checklist? Parsing invalid URL seems to be existing since a long time.

[slafortune]

@mananjadhav Yep, I agree with the regression test and that's been created. Is there a PR that introduced the bug has been identified?

[mananjadhav]

I couldn't trace an offending PR. Plus as I mentioned it looks like we've had the issue since inception.

[melvin-bot]

@mananjadhav, @slafortune, @jjcoffee, @jasperhuangg Whoops! This issue is 2 days overdue. Let's get this updated quick!