Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimise batch fetching of entities by their IDs #3142

Closed
muloem opened this issue Jul 24, 2019 · 1 comment
Closed

Optimise batch fetching of entities by their IDs #3142

muloem opened this issue Jul 24, 2019 · 1 comment
Assignees
@muloem
Labels
Backend
Milestone
1.9.49 Vibrant Vi...

Comments

@muloem
Copy link
Member

muloem commented Jul 24, 2019

When we have the IDs of multiple entities, we use a listByKeys(List) method to retrieve them. However, internally this method iterates of listByKey() this hitting the datastore with multiple requests for entities in a short span. We can improve this by making a single query with all the IDs.
@muloem muloem added the Backend label Jul 24, 2019
@muloem muloem added this to the 1.9.49 V... V... milestone Jul 24, 2019
@muloem muloem self-assigned this Jul 24, 2019
muloem added a commit that referenced this issue Jul 25, 2019
@muloem
[#3142] Make datanucleus classes available at compile time
43ee3b0 
* Needed in order to correctly compile the NucleusObjectNotFoundException class
muloem added a commit that referenced this issue Jul 25, 2019
@muloem
[#3142] Retrieve entities in a batch when a list of IDs is provided
0d8bbcd 
* This reduces the about of chatter between the datastore and the backend when retrieving multiple entities with known keys
muloem added a commit that referenced this issue Jul 25, 2019
@muloem
[#3142] Add missing import
f4a1c21
muloem added a commit that referenced this issue Jul 25, 2019
@muloem
[#3142] Update copyright year
dcd9545
muloem added a commit that referenced this issue Jul 26, 2019
@muloem
[#3142] Catch the exception when an object with a provided ID doesnt …
c016257 
…exist

* Fall back to the retrieval of items one-by-one incase one of the items is missing and causes an exception that prevents batch retrieval
valllllll2000 pushed a commit that referenced this issue Jul 29, 2019
Merge pull request #3144 from akvo/issue/3142-optmimise-batch-retrieval
146595a 
[#3142] Retrieve entities with known IDs in a single batch request
@muloem
Copy link
Member Author

muloem commented Aug 1, 2019

Test Plan

Covered by regression testing

@muloem muloem closed this as completed Aug 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@muloem muloem

Labels
Backend
Projects
None yet
Milestone
1.9.49 Vibrant Viburnum
Development

No branches or pull requests
1 participant
@muloem