-
Notifications
You must be signed in to change notification settings - Fork 1.3k
[core] Mark used offline region resources in batches #15521
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
@LukasPaczos would you be able to test this PR for Android SDK? |
The improvements are amazing! A small offline region that takes about 8s to download:
What's interesting, first re-download in the app process (for example after restarting the application when tiles are already in place) takes significantly longer than subsequent ones. |
Wow I didn't realise you could call mbgl-offline twice on the same output and it would just update, I always thought it would just replace it and re-download everything. Could we update this in the documentation? PS. I can't find any of the offline sideload documentation anymore, I guess it must have been removed on purpose. |
A region with Testing different batch sizes https://github.com/mapbox/mapbox-gl-native/pull/15521/files#diff-72890ff2093a9bfa6bfe3a9e8955d6dfR361: For batches of
|
Previously | Now | |
---|---|---|
First re-download in the current process | 41s | 2.7s |
Subsequent re-downloads in the same process | 38s | 1.3s |
For batches of 1000
resources
Previously | Now | |
---|---|---|
First re-download in the current process | 41s | 1.6s |
Subsequent re-downloads in the same process | 38s | 1.0s |
d208765
to
64de41b
Compare
We could make batch size adaptive, i.e. depend on the total resources count, I'll investigate it and implement in a follow-up PR. |
Wow, amazing work @pozdnyakov! |
Marking offline region resources "used" is on the hot code path for offline region download.
Previously, the engine attempted to update the database separetely for each resource and it caused a huge performance impact, especially when some tiles were already loaded.
Now, offline region resources are marked in batches (200 items at once).
Consider the time mesaurements of the
mbgl-offline
tool execution (Linux x64, release build):time ./mbgl-offline --north 41.4664 --west 2.0407 --south 41.2724 --east 2.2680 --output barcelona.db
without patch:
1st time:
2nd time (tiles are already loaded):
with the patch:
1st time:
2nd time (tiles are already loaded):
So, the proposed changes make the repeated load ~64 times faster 🎉