Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RTQC methods global attribute #78

Open
vturpin opened this issue Jul 21, 2022 · 5 comments
Open

RTQC methods global attribute #78

vturpin opened this issue Jul 21, 2022 · 5 comments
Labels
1.1 Tickets for )G 1.1 release
Milestone

Comments

@vturpin
Copy link
Member

vturpin commented Jul 21, 2022

"RTQC methods" and "RTQC methods doi" are currently mandatory in the global attribute and in the variable attribute.
I suggest we remove it from the global attribute and leave it mandatory in the variable attribute.

@castelao
Copy link
Member

Two parts:

  • I think it should be interchangeable. If the same method was applied for the whole dataset, why not use a global attribute one time? But if defined in the variable level, it would have priority. Let's consider a standard glider with a standard payload on the California Underwater Glider Network. I could define the standard QC procedure for the CUGN, and mention it only once, at the global level. One day, I do a special mission and add one extra sensor, I could still use the global definition, but for that specific variable, I would define an alternative approach. I think that this concept could be applied in other places. Actually, CF recommends exactly that principle.

  • The second point is that I don't think that QC should be mandatory but a priority. Ideally, the data provider should do the QC, but if they can't for any reason, we should allow the data to flow, be shared, and address the missing QC down the pipeline. I'm afraid of the situation of a small group collecting some data, willing to share, but not able to do it due to the lack of technical resources to implement such QC. I think we should guide the community on what would be ideal, but we should not impose it as a hard path.

@vturpin
Copy link
Member Author

vturpin commented Aug 29, 2022

Ok for both locations if this is a recommendation from Cf. But we need to explain that on the document.
Let's put it as highly desirable for RTQC method and doi.
QC is part of the role of the Data assembly centers as well as converting raw into OG1.0. DAC should have the ressources for RTQC.
Isn't there a risk to finally have non QC data sets ?

@castelao
Copy link
Member

Yes, there is a risk of having non-QCed data, but in my opinion, that is worse than having no data at all. Once a dataset gets into the system, somebody else (not ideal but) could apply the QC. From the user's perspective, the lack of QC method and flags tells enough how much that data should be trusted.

I do think QCing is important and should be a priority, but I'm afraid of making a harder entry point for new glider operators. I think highly desirable would be appropriate, no less than that. Any group with the required resources would certainly put effort into the high-priority components.

Could you open a PR with such changes on the document, please?

@vturpin
Copy link
Member Author

vturpin commented Sep 22, 2022

Can we put a level of requierment to a variable attribute?

@vturpin vturpin added the 1.0 For stuff that must be resolved before we are able to release 1.0 label Sep 22, 2022
@castelao castelao added this to the OG-1.0 milestone Sep 24, 2022
@justinbuck
Copy link
Collaborator

OG 1.0 meeting notes:
Global level RTQC is useful but would need to be a list of methods applied within the data
Variable level is needed by DACs.
Tool (software, in history attribute for ACDD), thresholds applied (history attribute for ACDD), and procedure (e.g. QUARTOD DOI) both need to be documented.
Needs to be downgraded to highly desirable in PG 1.0 (Gui working on pull request).
Needs more time and discussion and scoping in OG 1.1.

@justinbuck justinbuck added 1.1 Tickets for )G 1.1 release and removed 1.0 For stuff that must be resolved before we are able to release 1.0 labels Apr 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1.1 Tickets for )G 1.1 release
Projects
None yet
Development

No branches or pull requests

3 participants