Add CSV import flow to Categories page #47827

filip-solecki · 2024-08-21T23:24:40Z

Details

This PR add possibility to import Categories from spreadsheet. It uses new library called xlsx to read the data from files.

Fixed Issues

$ #47357

PROPOSAL: N/A

Tests

Log in to the app
Go to any Workspace -> Categories
Press three dots (in header)
Press Import spreadsheet
Press Choose file
Select xls, xlsx, csv or txt file
Select columns to import
Press import

Verify that no errors appear in the JS console

Offline tests

Feature not supported offline - info modal should be shown

QA Steps

Same as Test steps

Verify that no errors appear in the JS console

PR Author Checklist

I linked the correct issue in the ### Fixed Issues section above
I wrote clear testing steps that cover the changes made in this PR
- I added steps for local testing in the Tests section
- I added steps for the expected offline behavior in the Offline steps section
- I added steps for Staging and/or Production testing in the QA steps section
- I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
- I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
- I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).
I included screenshots or videos for tests on all platforms
I ran the tests on all platforms & verified they passed on:
- Android: Native
- Android: mWeb Chrome
- iOS: Native
- iOS: mWeb Safari
- MacOS: Chrome / Safari
- MacOS: Desktop
I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
I followed proper code patterns (see Reviewing the code)
- I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
- I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
- I verified that comments were added to code that is not self explanatory
- I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
- I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
  - If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:
- I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
- I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
- I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
- I verified the JSDocs style guidelines (in STYLE.md) were followed
If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
I followed the guidelines as stated in the Review Guidelines
I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
I verified that if a function's arguments changed that all usages have also been updated correctly
If any new file was added I verified that:
- The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory
If a new CSS style is added I verified that:
- A similar style doesn't already exist
- The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))
If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:
- I verified that all the inputs inside a form are aligned with each other.
- I added Design label and/or tagged @Expensify/design so the design team can review the changes.
If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

Android.mov

Android: mWeb Chrome

iOS: Native

ios_failed.mp4

ios_success.mp4

iOS: mWeb Safari

ios_safari.mp4

MacOS: Chrome / Safari

web.mov

MacOS: Desktop

desktop.mov

melvin-bot · 2024-08-22T14:47:29Z

@rushatgabhane Please copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

filip-solecki · 2024-08-22T14:48:40Z

@rushatgabhane - I think it is mostly ready for review, only on Chrome on Android there is an issue with ButtonWithDropdownMenu and I'll take care of this tomorrow
cc @rlinoz

shawnborton · 2024-08-22T15:40:11Z

Can we check the spacing of this empty state?

The illustration feels a bit close to the headline. And the text below the headline should use our textSupporting color. Please refer to the Figma for this one.

shawnborton · 2024-08-22T15:40:54Z

The border radius for the spreadsheet rows should be 8px, they look a bit bigger in your screenshots though.

shawnborton · 2024-08-22T15:42:21Z

When you open up the dropdown menu, I would think the currently selected menu option would have a selected BG color and a checkmark?

From the Figma:

package.json

src/components/ButtonWithDropdownMenu/index.tsx

src/components/Icon/Expensicons.ts

src/components/ImportColumn.tsx

src/components/FilePicker/types.ts

src/pages/workspace/categories/ImportCategoriesPage.tsx

src/components/OfflineWithFeedback.tsx

rlinoz

This is working very well overall!

rlinoz · 2024-08-22T16:09:05Z

src/CONST.ts

@@ -5496,6 +5499,30 @@ const CONST = {
            REMOVE: 'remove',
        },
    },
+
+    CSV_IMPORT_COLUMNS: {


There was the idea in the doc to split into constants for each file, should we update to match that?

I get the idea, but it is not needed IMO and in the function findColumnName we don't know it e.g. field GL Code is from categories or tags, so what should be set there as an attribute? (there is no difference, because they are the same) Example below:

case 'glcode': case 'gl': attribute = CONST.CSV_IMPORT_COLUMNS.CATEGORIES.GL_CODE; break;

src/libs/actions/Policy/Category.ts

src/pages/workspace/categories/ImportedCategoriesPage.tsx

mountiny

@filip-solecki couple small comments and the pending one from @dannymcclain

src/components/ImportColumn.tsx

src/components/ImportSpreadsheetColumns.tsx

src/libs/API/parameters/ImportCategoriesSpreadsheet.ts

mountiny · 2024-09-02T20:28:50Z

src/libs/importSpreadsheetUtils.ts

+function numberToColumn(index: number) {
+    let column = '';
+    let number = index;
+    while (number >= 0) {
+        column = String.fromCharCode((number % 26) + 65) + column;
+        number = Math.floor(number / 26) - 1;
+    }
+    return column;


A doc would be handy for this method, what exactly we are doing

dubielzyk-expensify · 2024-09-02T23:42:28Z

Agree that the selection should have a background color. As for the error, I guess it makes sense. I'm generally not a fan of allowing users to put them selves in a situation where validation issues occur as I prefer better defaulted options. But I think in this instance it might be weird to default the users to the first required value and better to let the user choose intentionally. The error message is also useful. Happy to go ahead with it

filip-solecki · 2024-09-03T07:25:41Z

@filip-solecki 🤔 Hmm, I just realized it has the background for me on the first open of the menu, but on subsequent opens, it does not have the background color. That's weird, right?

CleanShot.2024-09-02.at.09.51.14.mp4

Fixed!

mountiny

One NAB

src/libs/importSpreadsheetUtils.ts

Co-authored-by: Vit Horacek <36083550+mountiny@users.noreply.github.com>

filip-solecki · 2024-09-03T13:26:52Z

Screen.Recording.2024-09-03.at.15.25.37.mov

Now background of the selected item always has correct hex

dannymcclain · 2024-09-03T13:35:37Z

Nice!

mountiny

Thanks, the video looks good

github-actions · 2024-09-03T13:46:35Z

🧪🧪 Use the links below to test this adhoc build on Android, iOS, Desktop, and Web. Happy testing! 🧪🧪

Android 🤖	iOS 🍎
https://ad-hoc-expensify-cash.s3.amazonaws.com/android/47827/index.html	https://ad-hoc-expensify-cash.s3.amazonaws.com/ios/47827/index.html

Desktop 💻	Web 🕸️
https://ad-hoc-expensify-cash.s3.amazonaws.com/desktop/47827/NewExpensify.dmg	https://47827.pr-testing.expensify.com

👀 View the workflow run that generated this build 👀

mountiny · 2024-09-03T14:28:48Z

I have marked the snyk warnings about the xlms as resolved https://app.snyk.io/org/expensify/project/a75415c0-01a0-4906-abb8-070d86e05d58/pr-check/0bd42126-7fbf-4cea-a9f5-e8870001d371 I do not think we need to worry about those

OSBotify · 2024-09-03T14:47:42Z

✋ This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

OSBotify · 2024-09-03T23:42:48Z

🚀 Deployed to staging by https://github.com/mountiny in version: 9.0.29-0 🚀

platform	result
🤖 android 🤖	success ✅
🖥 desktop 🖥	success ✅
🍎 iOS 🍎	success ✅
🕸 web 🕸	success ✅

IuliiaHerets · 2024-09-04T10:57:36Z

This PR is failing for Desktop because of issue #48540

OSBotify · 2024-09-05T19:16:47Z

🚀 Deployed to production by https://github.com/roryabraham in version: 9.0.29-12 🚀

platform	result
🤖 android 🤖	success ✅
🖥 desktop 🖥	success ✅
🍎 iOS 🍎	success ✅
🕸 web 🕸	success ✅

ikevin127 · 2024-09-24T01:25:03Z

src/components/PopoverMenu.tsx

@@ -254,6 +259,7 @@ function PopoverMenu({
                            iconRight={item.iconRight}
                            shouldPutLeftPaddingWhenNoIcon={item.shouldPutLeftPaddingWhenNoIcon}
                            label={item.label}
+                            style={{backgroundColor: item.isSelected ? theme.activeComponentBG : undefined}}


Looks like this addition led to the following issue:

[CHECKLIST] [$250] Android & iOS - [Search v2.1] - Selected option is not highlighted #48543

Since given the new addition and the fact that in SearchTypeMenuNarrow.tsx we never pass the isSelected property. The issue was fixed by passing down isSelected which fixed the issue of the selected item highlight. More details in the issue proposal.

ZhenjaHorbach · 2024-10-29T07:59:49Z

src/components/ImportColumn.tsx

+    const columnValuesString = column.slice(containsHeader ? 1 : 0).join(', ');
+
+    const colName = findColumnName(column[0]);
+    const defaultSelectedIndex = columnRoles.findIndex((item) => item.value === colName);


Coming from this issue

In several cases defaultSelectedIndex will return -1 if the index is not found.
As result dropdown button does not match the selection in the dropdown list

And to fix this
A check was added If the index is not found, the default index will be 0

filip-solecki added 6 commits August 22, 2024 01:24

Add CSV import flow to Categories page

e407d21

Fix filePicker for native apps

e8a6e49

Add acceptable types to fix mWeb file selecting

e68b88c

Fix scrolling and import button position

2773a18

Fix footer styles

d5714f7

Fix duplicate column condition

da416c7

filip-solecki marked this pull request as ready for review August 22, 2024 14:47

filip-solecki requested review from a team as code owners August 22, 2024 14:47

melvin-bot bot requested review from rushatgabhane and removed request for a team August 22, 2024 14:47

rlinoz self-requested a review August 22, 2024 15:18

blazejkustra suggested changes Aug 22, 2024

View reviewed changes

This was referenced Aug 22, 2024

[Support CSV] Create import flow for members #47869

Closed

[Support CSV] Create import flow for tags #47870

Open

rlinoz requested changes Aug 22, 2024

View reviewed changes

filip-solecki added 9 commits August 23, 2024 08:06

CR fixes

4da476d

Remove unnecessary succesData update

41d1906

UI fixes

0b50987

Add GL Code field for control policies

4c7bc05

Merge branch 'main' into filip-solecki/import-categories-csv

def44cd

Remove unnecessary categories update on failure

6184709

Rename Onyx update method

9ae48f6

Prepare translations for other imports

861c7b6

Revert additional translations

f69f393

mountiny reviewed Sep 2, 2024

View reviewed changes

Fix selected bg and add docs

c68b384

mountiny previously approved these changes Sep 3, 2024

View reviewed changes

src/libs/importSpreadsheetUtils.ts Outdated Show resolved Hide resolved

Update src/libs/importSpreadsheetUtils.ts

77bbb95

Co-authored-by: Vit Horacek <36083550+mountiny@users.noreply.github.com>

filip-solecki dismissed mountiny’s stale review via 77bbb95 September 3, 2024 13:24

mountiny approved these changes Sep 3, 2024

View reviewed changes

mountiny merged commit 11952cc into Expensify:main Sep 3, 2024
18 checks passed

github-actions bot mentioned this pull request Sep 3, 2024

Deploy Checklist: New Expensify 2024-09-03 #48515

Closed

68 tasks

This was referenced Sep 4, 2024

[HOLD for payment 2024-09-24] [$125] Import categories - Error message for multiple GL code mappings displays "glCode" #48530

Closed

Import categories - Column is out of page when importing txt file #48536

Closed

IuliiaHerets mentioned this pull request Sep 4, 2024

[WAITING ON CHECKLIST - RUSHAT] [$250] Desktop - Categories - App crashes after importing categories and clicking Got it #48540

Closed

6 tasks

tgolen mentioned this pull request Sep 4, 2024

[Hold for payment 2024-09-12] [$250] Categories - GL code data disappears after importing same file that does not have GL code #48587

Closed

6 tasks

MonilBhavsar mentioned this pull request Sep 4, 2024

iOS - Import spreadsheet - Offline modal does not show up on iOS app only #48580

Closed

6 tasks

mallenexpensify mentioned this pull request Sep 11, 2024

[Support CSV] Create import flow for categories #47357

Closed

ikevin127 reviewed Sep 24, 2024

View reviewed changes

ikevin127 mentioned this pull request Sep 24, 2024

[CHECKLIST] [$250] Android & iOS - [Search v2.1] - Selected option is not highlighted #48543

Closed

6 tasks

ZhenjaHorbach reviewed Oct 29, 2024

View reviewed changes

ZhenjaHorbach mentioned this pull request Oct 29, 2024

[HOLD for payment 2024-10-29] [$250] Import categories - Dropdown button does not match the selection in the dropdown list #50312

Closed

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add CSV import flow to Categories page #47827

Add CSV import flow to Categories page #47827

filip-solecki commented Aug 21, 2024 •

edited

Loading

melvin-bot bot commented Aug 22, 2024

filip-solecki commented Aug 22, 2024

shawnborton commented Aug 22, 2024

shawnborton commented Aug 22, 2024

shawnborton commented Aug 22, 2024

rlinoz left a comment

rlinoz Aug 22, 2024

filip-solecki Aug 23, 2024 •

edited

Loading

mountiny left a comment

mountiny Sep 2, 2024

dubielzyk-expensify commented Sep 2, 2024

filip-solecki commented Sep 3, 2024

mountiny left a comment

filip-solecki commented Sep 3, 2024

dannymcclain commented Sep 3, 2024

mountiny left a comment

github-actions bot commented Sep 3, 2024

mountiny commented Sep 3, 2024

OSBotify commented Sep 3, 2024

OSBotify commented Sep 3, 2024

IuliiaHerets commented Sep 4, 2024

OSBotify commented Sep 5, 2024

ikevin127 Sep 24, 2024

ZhenjaHorbach Oct 29, 2024

Add CSV import flow to Categories page #47827

Add CSV import flow to Categories page #47827

Conversation

filip-solecki commented Aug 21, 2024 • edited Loading

Details

Fixed Issues

Tests

Offline tests

QA Steps

PR Author Checklist

Screenshots/Videos

melvin-bot bot commented Aug 22, 2024

filip-solecki commented Aug 22, 2024

shawnborton commented Aug 22, 2024

shawnborton commented Aug 22, 2024

shawnborton commented Aug 22, 2024

rlinoz left a comment

Choose a reason for hiding this comment

rlinoz Aug 22, 2024

Choose a reason for hiding this comment

filip-solecki Aug 23, 2024 • edited Loading

Choose a reason for hiding this comment

mountiny left a comment

Choose a reason for hiding this comment

mountiny Sep 2, 2024

Choose a reason for hiding this comment

dubielzyk-expensify commented Sep 2, 2024

filip-solecki commented Sep 3, 2024

mountiny left a comment

Choose a reason for hiding this comment

filip-solecki commented Sep 3, 2024

dannymcclain commented Sep 3, 2024

mountiny left a comment

Choose a reason for hiding this comment

github-actions bot commented Sep 3, 2024

mountiny commented Sep 3, 2024

OSBotify commented Sep 3, 2024

OSBotify commented Sep 3, 2024

IuliiaHerets commented Sep 4, 2024

OSBotify commented Sep 5, 2024

ikevin127 Sep 24, 2024

Choose a reason for hiding this comment

ZhenjaHorbach Oct 29, 2024

Choose a reason for hiding this comment

filip-solecki commented Aug 21, 2024 •

edited

Loading

filip-solecki Aug 23, 2024 •

edited

Loading