Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#2 - Fix user can submit 10-digit amount and gives an unexpected error #42815

Merged

Conversation

bernhardoj
Copy link
Contributor

@bernhardoj bernhardoj commented May 30, 2024

Details

When we select a currency without decimals, we can input up to 10 digit of amount which leads to a BE error. This PR fix it.

Fixed Issues

$ #42084
$ #42780
PROPOSAL: #42084 (comment)

Tests

Same as QA Steps

  • Verify that no errors appear in the JS console

Offline tests

Same as QA Steps

QA Steps

A.

  1. Open a submit expense page
  2. Change the currency to IQD
  3. Verify you can't enter an amount with more than 8 digit
  4. Select any participant and submit the expense
  5. Verify the request is successful and won't give an error

B.

  1. Go to workspace settings
  2. Go to Distance rates
  3. Click add rate
  4. Verify you can enter an amount with 3 digit decimals
  5. Input 0.655 and save the rate
  6. Click the new rate you just created
  7. Click Rate to open the rate amount edit page
  8. Verify the amount is shows 0.655
  • Verify that no errors appear in the JS console

PR Author Checklist

  • I linked the correct issue in the ### Fixed Issues section above
  • I wrote clear testing steps that cover the changes made in this PR
    • I added steps for local testing in the Tests section
    • I added steps for the expected offline behavior in the Offline steps section
    • I added steps for Staging and/or Production testing in the QA steps section
    • I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
    • I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
    • I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).
  • I included screenshots or videos for tests on all platforms
  • I ran the tests on all platforms & verified they passed on:
    • Android: Native
    • Android: mWeb Chrome
    • iOS: Native
    • iOS: mWeb Safari
    • MacOS: Chrome / Safari
    • MacOS: Desktop
  • I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
  • I followed proper code patterns (see Reviewing the code)
    • I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
    • I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
    • I verified that comments were added to code that is not self explanatory
    • I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
    • I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
      • If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:
    • I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
    • I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
    • I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
    • I verified the JSDocs style guidelines (in STYLE.md) were followed
  • If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
  • I followed the guidelines as stated in the Review Guidelines
  • I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
  • I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
  • I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
  • I verified that if a function's arguments changed that all usages have also been updated correctly
  • If any new file was added I verified that:
    • The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory
  • If a new CSS style is added I verified that:
    • A similar style doesn't already exist
    • The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))
  • If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
  • If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
  • If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
  • If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
  • If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:
    • I verified that all the inputs inside a form are aligned with each other.
    • I added Design label and/or tagged @Expensify/design so the design team can review the changes.
  • If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
  • If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native
Screen.Recording.2024-05-21.at.22.51.32.1.mov
Android: mWeb Chrome
Screen.Recording.2024-05-21.at.22.49.34.mov
iOS: Native
Screen.Recording.2024-05-21.at.22.52.23.1.mov
iOS: mWeb Safari
Screen.Recording.2024-05-21.at.22.55.05.mov
MacOS: Chrome / Safari
Screen.Recording.2024-05-21.at.22.40.12.mov
MacOS: Desktop
Screen.Recording.2024-05-21.at.22.41.23.mov

@bernhardoj bernhardoj requested a review from a team as a code owner May 30, 2024 03:14
@melvin-bot melvin-bot bot removed the request for review from a team May 30, 2024 03:14
Copy link

melvin-bot bot commented May 30, 2024

@ikevin127 Please copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

@melvin-bot melvin-bot bot requested a review from ikevin127 May 30, 2024 03:14
@bernhardoj
Copy link
Contributor Author

Additional recording showing the distance rate with 3 decimals (no trailing zero) is shown

Screen.Recording.2024-05-30.at.11.15.40.mov

@ikevin127
Copy link
Contributor

Reviewer Checklist

  • I have verified the author checklist is complete (all boxes are checked off).
  • I verified the correct issue is linked in the ### Fixed Issues section above
  • I verified testing steps are clear and they cover the changes made in this PR
    • I verified the steps for local testing are in the Tests section
    • I verified the steps for Staging and/or Production testing are in the QA steps section
    • I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
    • I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
  • I checked that screenshots or videos are included for tests on all platforms
  • I included screenshots or videos for tests on all platforms
  • I verified tests pass on all platforms & I tested again on:
    • Android: Native
    • Android: mWeb Chrome
    • iOS: Native
    • iOS: mWeb Safari
    • MacOS: Chrome / Safari
    • MacOS: Desktop
  • If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
  • I verified proper code patterns were followed (see Reviewing the code)
    • I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
    • I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
    • I verified that comments were added to code that is not self explanatory
    • I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
    • I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
    • I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
    • I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
    • I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
    • I verified the JSDocs style guidelines (in STYLE.md) were followed
  • If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
  • I verified that this PR follows the guidelines as stated in the Review Guidelines
  • I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
  • I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
  • I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
  • If a new component is created I verified that:
    • A similar component doesn't exist in the codebase
    • All props are defined accurately and each prop has a /** comment above it */
    • The file is named correctly
    • The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
    • The only data being stored in the state is data necessary for rendering and nothing else
    • For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
    • Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
    • All JSX used for rendering exists in the render method
    • The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions
  • If any new file was added I verified that:
    • The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory
  • If a new CSS style is added I verified that:
    • A similar style doesn't already exist
    • The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)
  • If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
  • If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
  • If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
  • If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
  • If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:
    • I verified that all the inputs inside a form are aligned with each other.
    • I added Design label and/or tagged @Expensify/design so the design team can review the changes.
  • If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
  • If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
  • I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native
android.mov
Android: mWeb Chrome
android-mweb.mov
iOS: Native
ios.mov
iOS: mWeb Safari
ios-mweb.mov
MacOS: Chrome / Safari
web.mov
MacOS: Desktop
desktop.mov

@bernhardoj
Copy link
Contributor Author

I have one question, currently, the max digit is 10.

In staging (or main), we can input max. 7 digits + 3 decimals for the distance rate. The max. digit before decimal (.) is 7.

But in this PR, we can input 8 digits + 2 decimals or 7 digits + 3 decimals. The max. digit before decimal (.) is 8.

But we can't input 8 digits + 3 decimals because the max digit is 10, do we want to increase the max digit to 11 for the distance rate?

I'm asking this because when I set the distance rate to 12345678.12, the BE returns 12345678.120, with a trailing zero, resulting in 11 digits.

@ikevin127
Copy link
Contributor

@marcochavezf @francoisl Any input on the comment above ?

return CONST.IOU.AMOUNT_MAX_LENGTH + 1;
}

return leadingZeroesLength + (absAmount === '0' ? 2 : absAmount.length);
return leadingZeroesLength + (absAmount === '0' ? 2 : stripDotFromAmount(absAmount).length);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, another way to fix this is by passing the removed-dot-amount to the old regex

const amountWithoutDecimalSeparator = stripDotFromAmount(absAmount);
if (/\D/.test(amountWithoutDecimalSeparator)) {
    return CONST.IOU.AMOUNT_MAX_LENGTH + 1;
}

Let me know if we want to use this instead.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I like the current version better, but can we also add unit tests to prevent future regressions please?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before adding a test, what do you think about my suggestion here. Do you think we should do it or keep it as it is for now?

If we want to do it, then we don't need the test for this specific function.

@ikevin127
Copy link
Contributor

ikevin127 commented May 30, 2024

@bernhardoj One of the reported deploy-blocker issues (#42780) is still happening on this PR:

Note: Same thing happens while online.

Screenshot 2024-05-29 at 20 35 35

@bernhardoj
Copy link
Contributor Author

It's happening on main too. That's weird because IQD doesn't have any decimal. We can't input 0.7 in submit expense amount page but we can in distance rate amount.

@bernhardoj
Copy link
Contributor Author

I found the root cause of the above issue, but I think it should be fixed on a separate issue.

We pass an extra 1 decimal to every currency

Then, we validate the amount, however, the BE returns the decimal with trailing zeros (0.700 for example)

if (MoneyRequestUtils.validateAmount(currentAmount, decimals, amountMaxLength)) {
return;
}
// If the changed currency doesn't support decimals, we can strip the decimals
setNewAmount(MoneyRequestUtils.stripDecimalsFromAmount(currentAmount));

so it's not valid because the extra decimal is 1, so the decimals is stripped.

@ikevin127
Copy link
Contributor

ikevin127 commented May 30, 2024

Interesting, I thought it's coming from our first PR somehow because the testers got back with this #42780 (comment) in which the issue doesn't seem to happen once the PR was reverted.

But I guess it's related to the IQD currency which might be different from MYR ? 🤔

Edit: I tested on current staging and MYR is fine now while offline but IQD edit is 0 regardless. So I think our PR might've caused the MYR to be 0 which is now fixed (after revert), but IQD is still 0 since that issue was present since before PR.

@bernhardoj
Copy link
Contributor Author

Yep, it depends with the currency because IQD doesn't support decimals.

Btw, regarding my question here, I just realized that if we increase the max digit to 11, we can then input 9 digits + 2 decimals which is invalid. So, we can't do that.

I propose that we refactor validateAmount a bit (maybe on another issue too?).

function validateAmount(amount: string, decimals: number, amountMaxLength: number = CONST.IOU.AMOUNT_MAX_LENGTH): boolean {
const regexString =
decimals === 0
? `^\\d+(,\\d*)*$` // Don't allow decimal point if decimals === 0
: `^\\d+(,\\d*)*(\\.\\d{0,${decimals}})?$`; // Allow the decimal point and the desired number of digits after the point
const decimalNumberRegex = new RegExp(regexString, 'i');
return amount === '' || (decimalNumberRegex.test(amount) && calculateAmountLength(amount, decimals) <= amountMaxLength);
}

The refactor will fully rely on the regex. So, we will remove calculateAmountLength. calculateAmountLength isn't powerful enough because it just count the digit length. We will update the regexString to validate the max digit of the amount before the decimal and the decimal max digit.

Currently, the regex matches this number: 123,123.456 but I don't see any usage in our app that allows the thousand separator(?) in the amount. So, we can update the regex to:

const regexString =
    decimals === 0
        ? `^\\d{1,${amountMaxLength}}$`
        : `^\\d{1,${amountMaxLength}}(\\.\\d{0,${decimals}})?$`;

amountMaxLength will be the max length before the decimal. Currently, amountMaxLength is the max length of all digits including the decimals. So for example, for distance rate, amountMaxLength will be 8 and decimals will be 3.

For tax rate, the max. value is 100%, so we can set the amountMaxLength to 3. This way, we can flexibly control the max. digit for different use cases.

Let me know what you think!

@francoisl
Copy link
Contributor

I'm asking this because when I set the distance rate to 12345678.12, the BE returns 12345678.120, with a trailing zero, resulting in 11 digits.

To clarify, what's the API command where the backend returns a decimal value with a trailing zero? As far as I know, in the backend we only try to deal with integers (money amounts in cents, essentially), to avoid rounding errors, so in that case it should have returned 1234567812.

Here's an example of UpdatePolicyDistanceRateValue if I set the rate to $1.23
image

@ikevin127
Copy link
Contributor

Please let me know besides the main issue (#42084) this PR fixes, if we're going to cover the other issues as well:

And if so I think it would be a good idea to update the PR description to include the other issues and Tests for each.
Otherwise, let me know what additional changes are going to be part of this PR besides the main issue (#42084).

@bernhardoj
Copy link
Contributor Author

@francoisl oh my bad, turns out it's the FE that formatted the amount to have 3 decimals points.

text: `${CurrencyUtils.convertAmountToDisplayString(value.rate, value.currency ?? CONST.CURRENCY.USD)} / ${translate(
`common.${customUnit?.attributes?.unit ?? CONST.CUSTOM_UNITS.DISTANCE_UNIT_MILES}`,
)}`,

function convertAmountToDisplayString(amount = 0, currency: string = CONST.CURRENCY.USD): string {
const convertedAmount = amount / 100.0;
return NumberFormatUtils.format(BaseLocaleListener.getPreferredLocale(), convertedAmount, {
style: 'currency',
currency,
minimumFractionDigits: getCurrencyDecimals(currency) + 1,

@francoisl
Copy link
Contributor

Ok yeah I think I like the idea in this comment to rely on a regex to validate the max number of digits and decimals.
Can we do it in this PR? It doesn't seem like a huge change in the end.

@bernhardoj
Copy link
Contributor Author

I can do it. I will update it on Monday.

@bernhardoj
Copy link
Contributor Author

Updated MoneyRequestUtils.validateAmount. Here is the list of the usages:

MoneyRequestConfirmationList (distance amount, 8 digits + 2 decimals):

distance.amount.mov

AmountForm
- AmountSelectorModal/AmountPicker/WorkspaceCreateTaxPage (create tax rate page, 4 digits + 4 decimals):

create.tax.rate.mov
- ValuePage (edit tax rate page, 4 digits + 4 decimals):
edit.tax.rate.mov
- PolicyDistanceRateEditPage (edit distance rate page 8 digits + 3 decimals):
edit.distance.rate.page.mov
- CreateDistanceRatePage (create distance rate page, 8 digits + 3 decimals):
create.distance.rate.page.mov
- RatePage (no usage)

MoneyRequestAmountInput
- MoneyRequestAmountForm
---- IOURequestStepAmount (IOU amount page, 8 digits + 2 decimals):

amount.page.mov
---- IOURequestStepTaxAmountPage (IOU tax amount page, 8 digits + 2 decimals)
tax.amount.page.mov
- MoneyRequestConfirmationList (individual split amount, 8 digits + 2 decimals):
individual.split.amount.mov

individual split amount input is the only one that formats the amount to have , separator, but when we edit it, the , will be removed or replaced with ..

const newAmountWithoutSpaces = MoneyRequestUtils.stripSpacesFromAmount(newAmount);
const finalAmount = newAmountWithoutSpaces.includes('.')
? MoneyRequestUtils.stripCommaFromAmount(newAmountWithoutSpaces)
: MoneyRequestUtils.replaceCommasWithPeriod(newAmountWithoutSpaces);

Added a unit test too.

@ikevin127
Copy link
Contributor

@francoisl Are you good with the latest changes and behaviour from #42815 (comment) ? Asking in order to confirm whether I should start working on the reviewer checklist.

@bernhardoj Follow-up regarding my #42815 (comment), besides the main issue attached in Fixed issues, assuming this PR's changes are addressing the other 2 issues, could you add reviewer / QA tests for those too ? Asking in order know what to test when I'm gonna start working on the reviewer checklist.

@bernhardoj
Copy link
Contributor Author

@ikevin127 updated the test step.

@ikevin127
Copy link
Contributor

I'll have the reviewer checklist ready by end of day today. @bernhardoj When you get a chance, please solve the conflicts.

@francoisl
Copy link
Contributor

@francoisl Are you good with the latest changes and behaviour from #42815 (comment) ? Asking in order to confirm whether I should start working on the reviewer checklist.

Woops sorry missed that comment – yes the changes in the videos look good to me.

@ikevin127
Copy link
Contributor

@bernhardoj I tested both scenarios and found some issues with scenario B, only with the IQD currency:

  1. Create WS and set default currency to IQD, then enable Distance rates and try to Edit rate (the default rate) -> edit field will show 0 (issue #42780, but for IQD only).
    ^ Additional observation: IQD currency allows 8 digits but only 1 decimal.

I think this is the last issue that needs to be addressed before I can ✅ Approve for final review.

@bernhardoj
Copy link
Contributor Author

It's the same as this one, right?

image

@ikevin127
Copy link
Contributor

Yes, but it only happens for IQD (and probably other similar currencies) and only when trying to edit the existing distance rates -> it's simply showing 0 instead of the rate with decimals.

@bernhardoj
Copy link
Contributor Author

Yeah, it happens with any currency that doesn't support decimals, such as yen. I thought we want to fix it on another issue based on this comment haha.

I'm not sure though what would be the best way to handle that. The currency doesn't support decimal, but in FE, we always add 1 extra decimal support, and BE accepts it too. Even the default distance rate has 2 decimals (0.67) for non-decimal (IQD) currency.

I think it needs to be discussed on another issue or if we want a temporary fix, we can set the distance rate decimal to always be 3, instead of currency decimal + 1 extra decimal.

cc: @francoisl

Copy link
Contributor

@ikevin127 ikevin127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the above context (#42815 (comment)) and the fact that the only issue left was/is already present on staging and was decided on as out of scope here, I'm approving with the mention that if the out of scope issue will surface post-merge, it will not be considered a regression of this PR.

@melvin-bot melvin-bot bot requested a review from marcochavezf June 7, 2024 06:12
Copy link
Contributor

@marcochavezf marcochavezf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I agree we should handle the edge case for currencies without decimal support as out-of-scope for now

@francoisl all yours

@francoisl francoisl merged commit 55015bd into Expensify:main Jun 7, 2024
19 checks passed
@OSBotify
Copy link
Contributor

OSBotify commented Jun 7, 2024

✋ This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

@OSBotify
Copy link
Contributor

🚀 Deployed to production by https://github.com/luacmartins in version: 1.4.81-11 🚀

platform result
🤖 android 🤖 failure ❌
🖥 desktop 🖥 success ✅
🍎 iOS 🍎 success ✅
🕸 web 🕸 success ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants