[Discuss] Local validation of data packages

[vitorbaptista]

Although goodtables.io is great, the feedback loop is too long, as I only know if my data is valid after it was pushed (or a PR created) and validated.

The solution I made on https://github.com/vitorbaptista/birmingham_schools/blob/master/tox.ini was to run goodtables locally (using tox). The same pattern can be extended to validate JSON files (not super useful, as goodtables already does this), lint scripts, etc.

WDYT?

[Stephen-Gates]

@vitorbaptista I'm flexible on this. As a non-programmer I was trying to keep it simple. I used this approach elsewhere and didn't mind it. I use the GitHub app, not Git to commit to repos. Was thinking that other contributors may be like me rather than developers.

Setting these things up has been an inhibitor to me contributing more. I typically bumble my way though installing the many tools used across OKI projects - often having no idea what I'm doing.

I don't mind what is used but I'll need help to set it up if it's any more complex.

[Stephen-Gates]

We should also explain how to use goodtables on branches. Is this possible or do we need frictionlessdata/goodtables.io#81 ?

[Stephen-Gates]

@vitorbaptista having just attempted to update the old packages I now 100% agree with your suggestion.

[vitorbaptista]

@Stephen-Gates The way I had set it up you just need Python and tox installed. This can be challenging for non-developers, as to run tox you need to go to the command line, but it shouldn't be too difficult. Also, as goodtables.io is set up, you don't need to use tox if you don't want to, it's just an extra tool.

[Stephen-Gates]

Thanks @vitorbaptista is that something you can add to this repo with some instructions?

I like the optionality

[Stephen-Gates]

Hi @vitorbaptista, I've attempted to add some tox bits in https://github.com/Stephen-Gates/example-data-packages/tree/add-tox

I've installed tox locally. What do I put in the command line to test a data package?
What changes are needed to test all data packages?

Your .gitignore looked complex - I assume I don't need all that?

I dropped some bits from Makefile that didn't seem required

Any help you can provide would be appreciated 😄

[rufuspollock]

Seems like a good item for a howto /cc @lwinfree @roll

[lwinfree]

Closing since framework allows for local validation

[roll]

https://framework.frictionlessdata.io/docs/guides/validation-guide#validating-a-package