Replace Pelican with Cardinal in pipeline (and make detailed coverage available)

[jpmckinney]

We are only using Pelican for field coverage, for which Cardinal is much faster.

We can store the output as part of the job, and make it available as part of the API in #268. We can also consider designing a report for the dataset's page, where a user can opt to view the detailed coverage.

We can then also use the output to either report:

• fields related to priority use cases, e.g. gender and sustainability (SPP)
• supported indicators (e.g. like in the usability notebooks)

[jpmckinney]

Edit: Moved tangential comment to #292

[jpmckinney]

From Pelican we get field counts and also some collection metadata. We can get the latter via an HTTP request to Kingfisher Process in the Process task's get_status method (once is_last_completed is true): open-contracting/kingfisher-process#421

[jpmckinney]

• Remove consume_exception=True from the request() call in in task/process.py, because it will no longer perform a presence check
• Eliminate parse_date, as Kingfisher Process returns dates in a normal format

[sentry-io]

Sentry Issue: REGISTRY-PELICAN-FRONTEND-B

[jpmckinney]

I linked a Sentry issue where a Pelican API request is quite slow on some collections (20s).

[jpmckinney]

We can maybe also (semi-)automate the additional_data fields – at minimum checking whether "No extensions or additional fields are used." is still true. We can also check that a declared extension is actually used (we shouldn't report extensions unless the fields they define are used).

[jpmckinney]

We can store the output as part of the job, and make it available as part of the API in #268

Once that's done, we can simplify the field list notebook: https://github.com/open-contracting/notebooks-ocds/blob/main/component_get_field_list_all_registry.ipynb

[yolile]

Once that's done, we can simplify the field list notebook

Or, add a button somewhere in the data registry to export the list of fields from all publications (if there is demand for that)