-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Datasets consistency #336
Datasets consistency #336
Conversation
Could we don't put an option Personally I can't think of a reason why I would rather have seconds over an array of datetime object. |
Because different datasets encode their time differently. For example, GLAD time is encoded as integer minutes since some time in 2012 (perhaps the start of deployment). It's possible to do it with After working some time with |
I see the issue. One thing to also weight in is if someone is used to work with one of those dataset, which encode the time in days, I'm not sure it's ideal for us to ingest it but then convert the time to seconds. My preferences would be:
|
I tend to agree. What do you think about this approach:
I'm only unsure of this keyword parameter name. |
I think the questions boil down to:
The source encoding of the dataset may be somewhat arbitrary and not necessarily about how data was stored in the upstream data, but how it was processed in the adapter. E.g. in case of GLAD, it's entirely due to how the datetime stamps in text files were ingested into Pandas DataFrame and then converted to Xarray Dataset. @selipot chime in please on the 2 points above. |
For point 1 above I like the idea of having a |
There are conflicts due to the versions. I probably created this, my apologies. |
OK, so it sounds like we'll go with
I think that's a fine approach to move forward with here. |
I think the software version in |
The version in the main branch is incorrect. This PR bumps from the current 0.28.0 to 0.29.0. |
This PR:
ids
and appliesid(traj)
as a coordinate to GDP hourly and 6-hourly (closes modify the gdp ragged array #326)id_var_name
from"ID"
to"id"
inragged.subset
.Updates docs/usage.rst to reflect these changes.Adapters are not affected. If time needs to be converted at the adapters level, please use
clouddrift.datasets._to_seconds_since_epoch()
.