Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alternative time variales incorrectly processed #47

Open
callumrollo opened this issue Dec 16, 2021 · 2 comments
Open

Alternative time variales incorrectly processed #47

callumrollo opened this issue Dec 16, 2021 · 2 comments
Labels
bug Something isn't working

Comments

@callumrollo
Copy link
Collaborator

@Maribaken discovered a bug when trying to process AD2CP_TIME from a SeaExplorer. This is timing from the ADCP internal clock, which can differ substantially from the SX payload clock, so it's important to carry this data forward.

These are correctly interpreted as datetimes at the raw_to_rawnc stage

:array([ 'NaT', 'NaT',
'2021-12-03T10:45:39.000000000', '2021-12-03T10:46:09.000000000',
'2021-12-03T10:46:39.000000000'], dtype='datetime64[ns]')

but get converted to floats during raw_to_L0timeseries becoming

array([ nan, nan, 0., ..., 1990., 2020., 2050.])

The culprit is the decode_times=False kwarg in sensor = xr.open_dataset(f'{indir}/{id}-{kind}pld.nc', decode_times=False) in seaexplorer.py line 248

I think this flag is in here to force the main timestamp to be in seconds since 1970, so that the function utils.get_profiles_new works as it takes calculates a median time difference between samples in seconds.

Should I do a specific fix for the AD2CP? Are there other sensors that record internal time we need to keep? Should I try to rework utils.get_profiles_new to work with datetimes rather than seconds since 1970?

@callumrollo callumrollo added the bug Something isn't working label Dec 16, 2021
@jklymak
Copy link
Member

jklymak commented Dec 16, 2021

Probably we should try to update it all to work properly with datetime64 objects. Probably when I wrote this, I wasn't up on how to use dataetim64 properly.

@callumrollo
Copy link
Collaborator Author

Cool, I'll have a crack. I still suck at datetime64 tbh but it is a better standard for sure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants