Description
Sometimes we have cases, when datasets pass the Data Package validation and are published, but still throwing 500 errors.
Tasks
- discuses the pros and cos for all available options (see below) or suggest new, better ones
- decide which option better fits us (see solutions part)
- Implement
Analysis
Reason for this is that even after validation there still may remain beyond Frictionless metadata specifications. Eg licenses
property should be list of objects, not directly object etc and DP validation does not cover this kind of "mistakes" from users. Problem is that on server side we follow "rules" and always waiting that metadata (datapackage.json) will follow specifications.
We faced issue exactly like this with Geo data packages - you can see deeper discussions here #335
The problem is that, we can not rely on user and assume he/she will always set licenses to be a list of the objects, bot not either "overload" and ask to learn all the specifications.
We have several options:
- have the flag inside jinja templates and check for type.
- Modify on server side - have helper function that checks if it is list and appends to list if not
- Modify the validation script, so that it throws warning/error if
licenses
is not set according to specs. This way we are guaranteed that it will always be list of the objects
Personally I think most reliable way to avoid this kind of error in the future is 3rd option (to modify validation script and worn users before publishing). But this is from perspective of developer.
We should consider that users do not want to sit and learn all the tiny details. They wan't to publish their data and that should be as simple as possible
Solution
As right now the 500 error might caused only in because Jinja template is expecting zeroth element from object Eg:
<div class="license">
<i class="fa fa-gavel"></i> <a href="{{dataset.licenses[0].url}}" title="Available under the following License">{{dataset.licenses[0].name or dataset.licenses[0].id}}</a>
</div>
I propose to have a helper function in logic layer validate_for_template(descriptor)
that will check type of licenses and just pop licenses from descriptor if not valid