-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tool to migrate data between SpatialData versions #680
Comments
Thanks for tracking this here. An alternative to consider opening some GitHub discussions for less common issues, as done here: #657. For more common conversion steps, like migrating from the |
Current format changes:
Current versioning support in SpatialData:
Types of changes in SpatialData format:
Existing tools:
Aims:
Requirements:
|
Thanks for the detailed summary of the format changes. I would proceed as follows. I would not include these three points in a migration tool:
For the example above this would roughly be: "Open a terminal and move
I think that having a migration tool dealing with the first 3 problems would be complex and it's better to explain to the user what the problem is and how to build a solution, so that they know what happens and they can choose a solution that suits them. For instance, there is no canonical way to fix missing radii because they were negative, so I'd let the user manually choose how to address them. What do you think about this way to proceed? |
super useful summary @aeisenbarth , I also agree with @LucaMarconato that a separate tool is maybe too much of an overkill. I think the best way would be to stick to the format version as much as possible, and reflect this in the code, without changing the API.
I think this ideally would be true, we should strive to make this possible imho. |
New format changes:
|
Is your feature request related to a problem? Please describe.
When the specification of SpatialData is changed, existing datasets do not have these changes, and under circumstances may even become incompatible.
For the in-memory representation, the library's reader functions support reading older versions (in most cases). However, users may want to upgrade the on-disk data to the latest version. Another case is when something is not covered by backward-compatibility of readers, e.g. due to errors in the data (#655).
Describe the solution you'd like
As discussed earlier, we want a tool for migrating data, if more format changes occur in future. I open this new issue for tracking this, since #655 was too specific and is closed.
Describe alternatives you've considered
write
to save in the latest format: Some cases may not be covered by readers.spatialdata
library: Migration is not a frequent use case and would bloat the library, especially if supporting special cases (one-time issues, erronous data). Additionally, backward-compatibility of certain features can be easily deprecated in the library, while still preserving it in a separate tool.spatialdata-migrate
toolThe text was updated successfully, but these errors were encountered: