You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Aug 2, 2021. It is now read-only.
The http reader is suboptimal since it downloads in syncronous batches translating to range queries for the LazyChunkReader. Previously we optimised for buffersize in this reader, however, this still leaves longer latencies since each batch is blocking until the slowest chunk arrives.
A low hanging fruit to improve is to extend this reader with an asyncronous lookahead that brings concurrent network retrieval across batches and thereby significantly reduce download latencies.
The text was updated successfully, but these errors were encountered:
This should not be in here. The specifications need to go into a SWIP the timeline is a result of the issues which are assigned to the user-story. In This case it's the 200 MB / 500 MB iterative file-size support increase and the improvement on stability.
The http reader is suboptimal since it downloads in syncronous batches translating to range queries for the
LazyChunkReader
. Previously we optimised for buffersize in this reader, however, this still leaves longer latencies since each batch is blocking until the slowest chunk arrives.A low hanging fruit to improve is to extend this reader with an asyncronous lookahead that brings concurrent network retrieval across batches and thereby significantly reduce download latencies.
The text was updated successfully, but these errors were encountered: