Downloading an offline pack blocks the visible map from loading

[friedbunny]

If the active style of a map view is changed while an offline pack is downloading, the new style will fail to load (i.e., it only shows a black screen) while that download is active.

1. Start downloading an offline pack
2. Switch the visible style on your map view
3. The new style will not load until the offline pack finishes (or pauses) downloading

/cc @jfirebaugh @1ec5 @zugaldia

[friedbunny]

This will also prevent the current map from loading (e.g., viewDidLoad), if you start the offline download before it has a chance to finish loading.

Offline download (or pack verification) in progress

Offline download finished, map finally loads

[jfirebaugh]

This occurs because OnlineFileSource has a single FIFO queue for requests, which OfflineDownload basically floods with all the tile requests at once. So not only will an offline download prevent a new style from loading until the download is complete, it'll prevent new tiles from loading as well.

The options for fixing this are:

• Use a priority queue in OnlineFileSource and give interactive requests higher priority.
• Change OfflineDownload so that it does its own metering of requests.
• Revisit OfflineManager and MapView should share a singleton DefaultFileSource #4386; i.e. switch back to having distinct OnlineFileSources for offline downloads and online activity.

The second option sounds most attractive to me. To reduce memory usage, we know we want to change the OfflineDownload implementation anyway so that it requests tiles in a streaming fashion, rather than all in one shot, and metering would fit into that.

[halset]

@jfirebaugh Any tips on how to get going on the second option?

I have tried to use a mutex and a condition_variable to control the number of concurrent requests from the OfflineDownload, but it does not work as the requests are created and executed on the same thread using the RunLoop.

Issuing "util::RunLoop::Get()->runOnce();" after each ensureResource will at least keep the queues small, but it does not improve the interactivity of the map by much.

[halset]

(Sorry for the noise, but I am grateful for any help on this subject, @friedbunny @incanus.)

[jfirebaugh]

At minimum, the loop over tiles and the loop over glyphs need to be made incremental and fully asynchronous. These are the two loops that really flood OnlineFileRequest's own internal queue, delaying requests triggered by user interactions. Therefore rather than calling ensureResource in a loop for every required resource, ensure only a limited number, keep the rest in a queue, and draw from the queue when a prior request completes.

I'm not sure whether it makes sense to fully generalize the request queuing mechanism to the other resource types as well (style, sources, and sprites), or whether that would add too much complexity. You would probably have to track more state explicitly as class members, because it's no longer tracked implicitly via the stack.

As noted, OnlineFileSource has a similar internal queuing mechanism already. On one hand, we should consider if there are any possibilities for code reuse. On the other, OnlineFileSource is suffering from bugs in its state management (#5827), so beware, and try for something simpler.

[Duckbadger]

Hi @jfirebaugh, any update on this?
I've used both version 3.3.4 and alpha 4 of 3.4.0, and calling suspend still has to wait until the queue has finished checking previously downloaded resources.
It's a huge blocker at this point because the map view will not load any tiles until that task is suspended, which is often a large amount of time.

[jfirebaugh]

It's fixed for the next release (3.4.0 alpha5).

[Duckbadger]

Okay thanks, we'll check it out when alpha 5 is available.