Compile Release Transform: raise MissingDateKeyError('The date field of at least one release is missing.')

[odscjames]

We should at least make sure such errors don't crash the whole Python process - but how should the app handle data with no date?

[jpmckinney]

What do you think the behavior should be, @yolile @romifz @mrshll1001?

The problem is that, if the date is missing, then the release can't be ordered correctly for the merge routine.

[romifz]

I think these releases could be ignored. Are transformation warnings reported somewhere? It would be nice to have a place to look for these.

[jpmckinney]

For your question, that's the subject of #242

[odscjames]

Compile Release Transform: date missing

[odscjames]

As this is causing crashes, I'm going to do a quick fix under the small work rule. I'll just get it to drop whole OCID's if "date" is missing with a note. This is a bit of an extreme option but at least it won't be crashing and other data will be processed.

[odscjames]

Deployed quick fix. Leaving open to work out if better way to process?

[jpmckinney]

@romifz Does it make sense to not produce any compiled release for an OCID, if any of its individual releases lacks a date? Or is it preferable to create a compiled release with the individual releases that have a date? I don't have a strong opinion; it's a choice between missing processes and missing releases.

cc @duncandewhurst as not tagged earlier due to leave.

[romifz]

I think the second option may be better. When doing a review we want to have as many data available as possible. If we exclude processes there is a small chance of missing an issue we can report on.

[duncandewhurst]

I think the second option may be better. When doing a review we want to have as many data available as possible. If we exclude processes there is a small chance of missing an issue we can report on.

Agreed. We should also add a step to our standard notebook set up to check for these errors. I've added a note to crm issue #3918

[odscjames]

Looking at this. Still good to do quick fix tho, as that will catch any errors OCDS Merge throws, not just date ones.

[odscjames]

Deployed new process - will merge with date ones and alert. Think this is now done.