Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use faster implementation for merging #186

Merged
merged 16 commits into from
Oct 5, 2022

Conversation

Szymon20000
Copy link
Contributor

The whole problem is described here Expensify/App#11185

On Android merging takes a lot of time
I Initially thought that maybe it's caused by slow loads implementation but after looking more deeply into the problem it looks like require takes in lodash a lot of time. Have no idea how because it should be required only once and later not take time at all but it's not the case. Maybe some circular dependencies?

I used the latest react-native-onyx package .18 with recent fixes like getting rid of merging items of array one by one. But it seems to no solve the problem.

Screenshot 2022-09-28 at 10 54 26

Screenshot 2022-09-28 at 10 54 50

Screenshot 2022-09-28 at 10 57 39

Screenshot 2022-09-28 at 10 57 33

Details

Related Issues

GH_LINK

Automated Tests

Linked PRs

@Szymon20000 Szymon20000 requested a review from a team as a code owner September 28, 2022 08:58
@github-actions
Copy link
Contributor

github-actions bot commented Sep 28, 2022

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@melvin-bot melvin-bot bot requested review from AndrewGable and removed request for a team September 28, 2022 08:59
@Szymon20000
Copy link
Contributor Author

@iwiznia Could you take a look as you recently optimised the same part?

Copy link
Contributor

@mountiny mountiny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Szymon20000 You will also need to sign the CLA in this repo as well!

Regarding the JSDoc, lets update the indentation as you can see here, for example https://github.com/Expensify/App/blob/da6e3aac5b64154e1d87c261f34829e097c7b433/src/libs/ReportUtils.js#L388-L393

ie the next line has one whitespace at the start of the line. Thanks!

lib/fastMerge.js Outdated Show resolved Hide resolved
lib/fastMerge.js Outdated Show resolved Hide resolved
lib/fastMerge.js Outdated Show resolved Hide resolved
lib/fastMerge.js Outdated Show resolved Hide resolved
lib/fastMerge.js Outdated Show resolved Hide resolved
lib/fastMerge.js Outdated Show resolved Hide resolved
Szymon20000 and others added 3 commits September 28, 2022 16:31
Co-authored-by: Vit Horacek <36083550+mountiny@users.noreply.github.com>
Co-authored-by: Vit Horacek <36083550+mountiny@users.noreply.github.com>
@Szymon20000
Copy link
Contributor Author

I have read the CLA Document and I hereby sign the CLA

Copy link
Contributor

@mountiny mountiny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Szymon20000 I am also thinking if we should update the Docs for the functions, they always just copy what the function name is so it is not very useful. So we should either expand on it (why it is there/what it is used for - which would be handy imho) or remove them as they dont add any value as its current form

lib/fastMerge.js Outdated Show resolved Hide resolved
lib/OnyxCache.js Outdated Show resolved Hide resolved
lib/OnyxCache.js Outdated Show resolved Hide resolved
lib/fastMerge.js Outdated Show resolved Hide resolved
Copy link
Contributor

@mountiny mountiny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have some failing test still:
Onyx › Cache Service › merge › Should do nothing to a key which value is undefined``

mountiny
mountiny previously approved these changes Sep 30, 2022
Copy link
Contributor

@mountiny mountiny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Lgtm!

lib/fastMerge.js Outdated Show resolved Hide resolved
@Szymon20000 Szymon20000 requested review from tgolen and removed request for AndrewGable October 3, 2022 07:30
@luacmartins luacmartins self-requested a review October 3, 2022 14:35
Copy link
Collaborator

@tgolen tgolen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good, but I think the ESlint warnings need to be cleaned up better. The only time an eslint rule should be disabled is when there is a business reason for it (like performance). Otherwise, you should try to follow them rather than disable them.

lib/Onyx.js Outdated Show resolved Hide resolved
lib/fastMerge.js Outdated Show resolved Hide resolved
lib/fastMerge.js Show resolved Hide resolved
destination[key] = cloneIfNecessary(source[key], optionsArgument)
} else {
destination[key] = merge(target[key], source[key], optionsArgument)
// eslint-disable-next-line rulesdir/prefer-underscore-method
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason not to use underscore here? If it's because of performance, please add a code comment explaining that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does the function has to be defined as mergeObject = function (target, source) instead of simply function mergeObject(target, source)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have to do that otherwise we would use it before it's defined, since we have the following circular dependency mergeObject -> fastMerge -> mergeObject

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when 2 functions are on the same level, they are registered at the same time, it's just that eslint would flag it as a code style issue, because our code style is to define stuff before it's used - impossible to address in case of a circular dependency, but we can just disable the warning

The actual problem is with variables - you can't use a variable before it is defined, but you can use a function before it's defined, because functions are always registered before any code runs

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying @kidroca. In that case, I agree that we should declare it as function mergeObject() and disable the lint warning with a comment explaining why

lib/fastMerge.js Outdated Show resolved Hide resolved
lib/storage/providers/LocalForage.js Show resolved Hide resolved
lib/Onyx.js Show resolved Hide resolved
lib/OnyxCache.js Show resolved Hide resolved
destination[key] = cloneIfNecessary(source[key], optionsArgument)
} else {
destination[key] = merge(target[key], source[key], optionsArgument)
// eslint-disable-next-line rulesdir/prefer-underscore-method
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

lib/fastMerge.js Show resolved Hide resolved
lib/fastMerge.js Outdated Show resolved Hide resolved
mountiny
mountiny previously approved these changes Oct 4, 2022
Copy link
Contributor

@mountiny mountiny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the comments, looks good to me

luacmartins
luacmartins previously approved these changes Oct 4, 2022
Copy link
Contributor

@luacmartins luacmartins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few NAB comments

lib/fastMerge.js Outdated Show resolved Hide resolved
lib/Onyx.js Outdated Show resolved Hide resolved
lib/OnyxCache.js Outdated Show resolved Hide resolved
lib/fastMerge.js Outdated Show resolved Hide resolved
Copy link
Collaborator

@tgolen tgolen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry to keep harping on the style, but we are sticklers for consistency. Please try to follow previous patterns and our style guides https://github.com/Expensify/Style-Guides/blob/main/javascript.md

lib/Onyx.js Outdated Show resolved Hide resolved
lib/fastMerge.js Outdated
Comment on lines 5 to 10
/**
* Is the object Mergeable
*
* @param {*} val
* @returns {*|boolean}
*/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/**
* Is the object Mergeable
*
* @param {*} val
* @returns {*|boolean}
*/
/**
* @param {mixed} val
* @returns {mixed}
*/

Don't include a method description unless it's valuable (and what you had there just repeats the method name). For the return value, maybe we should cast it to always return a boolean?

lib/fastMerge.js Outdated Show resolved Hide resolved
lib/fastMerge.js Outdated Show resolved Hide resolved
lib/fastMerge.js Outdated Show resolved Hide resolved
lib/storage/providers/LocalForage.js Outdated Show resolved Hide resolved
Co-authored-by: Tim Golen <tgolen@gmail.com>
@Szymon20000 Szymon20000 dismissed stale reviews from luacmartins and mountiny via 3ba5681 October 5, 2022 11:35
Copy link
Contributor

@mountiny mountiny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the style changes 👍

Copy link
Collaborator

@tgolen tgolen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Thank you for the changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants