Unify Dataset IO #535
Unify Dataset IO #535
Conversation
…-datasetio"" This reverts commit 4c67c0a
There was a problem hiding this comment.
Where are the changes in xcube.webapi.context?
Also I don't understand your comment:
When reviewing this, please consider especially the function get_data_store_pool of the ServerContext: This function used to remove a data store pool if there were no datastores explicitly configured. I had to change this, but I don't fully understand what was the original reasoning behind this.
I cannot see what you are referring to.
In get_data_store_pool, there were the following lines:
|
| }, | ||
| { | ||
| 'Identifier': 'z_4', | ||
| 'FileSystem': 'obs', |
There was a problem hiding this comment.
I think we should stop using "obs" when referring to s3 object storage - and exchange it to "s3" as used in the DataStores configurations.
There was a problem hiding this comment.
Correct - but for now, we'll move this to another PR.
…tasetio2 # Conflicts: # xcube/webapi/context.py
Codecov Report
@@ Coverage Diff @@
## master #535 +/- ##
==========================================
+ Coverage 91.63% 91.77% +0.14%
==========================================
Files 292 292
Lines 26587 26742 +155
==========================================
+ Hits 24362 24543 +181
+ Misses 2225 2199 -26
Continue to review full report at Codecov.
|
This PR addresses #516. It ensures that, if a Configuration with Datasets is passed to a ServiceContext, each dataset is assigned to a datastore that will be used to open the data (unless the dataset is in-memory). It is attempted to find common roots for data stores to group datasets located at same/similar locations together.
When reviewing this, please consider especially the function get_data_store_pool of the ServerContext: This function used to remove a data store pool if there were no datastores explicitly configured. I had to change this, but I don't fully understand what was the original reasoning behind this.
This is a follow-up PR to erroneously merged PR #527 .
Checklist:
[ ] Add docstrings and API docs for any new/modified user-facing classes and functions[ ] New/modified features documented indocs/source/*CHANGES.mdRemember to close associated issues after merge!