-
Notifications
You must be signed in to change notification settings - Fork 295
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fallback when reading sharded data from a host that does not support range requests #619
Comments
I can imagine that in some cases this could be useful. But on the other hand, it isn't clear how often it would really be useful --- I imagine in many cases having to download the entire shard would lead to a very poor experience if it fits in memory at all. Certainly if this were supported, it would be important to surface a warning to the user somehow, to indicate that it is operating in a degraded mode and that they should fix the web server. A simpler improvement would be to surface a more informative error when range requests aren't supported but e.g. a HEAD request succeeds. Related to this, it would be useful to surface the chunk download errors outside of the javascript console... |
Yes, the net utility of falling back to fetching the whole shard depends on the empirical question of what the typical shard size is. If nearly everyone deploys 10 GB shards, then it's not practical to consider fetching that. But 100 MB shards, on a local connection, would be technically feasible, albeit not a great user experience. |
Apropos #606, not every host supports byte range requests to the same degree (or at all). When fetching chunks from a sharded datasource, if byte range requests fail, Neuroglancer displays no data at all. But in this case Neuroglancer could fall back to fetching the entire shard file and reading chunks in memory. This will fail for large shards, but with the same familiar failure mode triggered by any object that's too large to fetch. For small shards, however, fetching a full shard as a fallback would allow neuroglancer to display data that it otherwise could not.
The text was updated successfully, but these errors were encountered: