enhancement: make possible to run tests and related stuff using tox

[ssato]

Make possible to run tests (python setup.py test) and other related
stuff (flake8, doc8 and yamllint) using tox.

Signed-off-by: Satoru SATOH satoru.satoh@gmail.com

[coveralls]

Coverage remained the same at 97.867% when pulling 11d90dc on ssato:tox into 597e88b on adrienverge:master.

[ssato]

I think that it doesn't break anything and just makes our hacking life easier by adding a feature to allow us testing it using just 'tox -e py36' command for example. So could you please take just a little time to looks this? I'm very happy if it's merged.

[adrienverge]

I understand it would save time for people that use tox, but not for others, and I try to keep the repo as simple as possible (for many reasons, including easy maintenance and easy bootstrapping for non-Python users).

I'm not favorable to this, sorry.

[ssato]

I understand it would save time for people that use tox, but not for others, and I try to keep the repo as simple as possible (for many reasons, including easy maintenance and easy bootstrapping for non-Python users).

I'm not favorable to this, sorry.

I understand you want to keep it simple, however, let me try to explain why it's valuable and necessary for developers from my point of view, please.

• Pros

  • Tox looks a de facto testing tool widely used and developers might be familiar with it already, I guess.

  • Tox is just needed for testing and users of yamllint are not affected at all by adding its dependency.

  • Tox does not only save time but also break down the barrier to develop and test yamllint.

  • Because runtime and test time dependencies are written in each dependent files separately, you can update them without affecting others as needed.

  • By using tox, you can simplify https://github.com/adrienverge/yamllint/blob/master/CONTRIBUTING.rst a lot; just ask developers to run tox should be enough.

  • Because my change just adds some files, that does not bring any risks, AFAIK.

  • Tox enables us testing in multiple virtual environments different python versions are installed at once easily without any specific CI services.

    • If my understanding is correct, you're using Travis-CI for this purpose but other developers cannot test it easily because it depends on Travis-CI. Also because the test time dependencies and steps needed for testing are hard-coded into the configuration file for Travis-CI, it's difficult to use other CI services as an alternative or at the same time with consistencies are kept or test it locally w/o any such CI services.

      • Here are some such example configurations using tox + {Travis-CI, GitLab Actions} resolve these issues. You see that the parts depends on CI services are kept to minimum. Of course, if you want, I'll try to arrange such configurations for yamllint.

        • tox.ini: https://github.com/ssato/ansible-lint-custom-rules/blob/master/tox.ini
        • Travis-CI configuration: https://github.com/ssato/ansible-lint-custom-rules/blob/master/.travis.yml
        • Github Actions CI: https://github.com/ssato/ansible-lint-custom-rules/blob/master/.github/workflows/tests.yml

• Cons

  • maintainance and learning costs: Although tox's configuration is very simple, it brings another test time dependency and force us to learn how to use it.

[ssato]

I opened another PR for this.