refactor: Various tweaks and improvements

Added detection for github and gitlab so documentation deployment can be adjusted accordingly.

Reference:
https://gitlab.com/pages/mkdocs
https://squidfunk.github.io/mkdocs-material/publishing-your-site/#gitlab-pages
https://docs.gitlab.com/ee/user/project/pages/getting_started_part_one.html
https://medium.com/faun/automate-you-documentation-with-gitlab-and-mkdocs-26e1b564f9f2

Updated to add python 3.9 to tests and to also setup gitlab pages on deploy stage.
Fixed hardcoded github.io even when gitlab is selected.
Set this for gitlab documentation pages: gitlab.io.

Added precommit hook so that all tests are run via poetry run duty test.

https://pre-commit.com/#intro

Setup coverage to be extracted when coverage is run
https://docs.gitlab.com/ee/ci/yaml/README.html#coverage
coverage: '^TOTAL.+?(\d+\.\d+\%)$'

Added explicit 'tests' path to pytest config.
It helps IDEs like PyCharm finding tests.

https://docs.pytest.org/en/stable/customize.html#pytest-ini

project/.gitlab-ci.yml

@@ -11,6 +11,7 @@ cache:
 stages:
   - quality
   - tests
+  - deploy
 
 .install-deps-template: &install-deps
   before_script:
@@ -27,7 +28,10 @@ stages:
 
 check-docs:
   <<: *quality
-  script: poetry run duty check-docs
+  script:
+    - mkdir -p build/coverage 
+    - touch build/coverage/index.html 
+    - poetry run duty check-docs
 
 check-code-quality:
   <<: *quality
@@ -59,3 +63,28 @@ python3.7:
 python3.8:
   <<: *test
   image: python:3.8
+
+python3.9:
+  <<: *test
+  image: python:3.9
+
+
+pages:
+  stage: deploy
+  <<: *install-deps
+  image: python:3.6
+  dependencies:
+  - python3.6
+  - python3.7
+  - python3.8
+  - python3.9
+  script:
+  - poetry run duty test
+  - poetry run duty coverage
+  - poetry run duty docs
+  coverage: '/^TOTAL.+?(\d+\.\d+\%)$/'
+  artifacts:
+    paths:
+    - public
+  only:
+  - master

project/.pre-commit-config.yaml

@@ -0,0 +1,28 @@
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v3.2.0
+    hooks:
+    -   id: trailing-whitespace
+    -   id: end-of-file-fixer
+    -   id: check-yaml
+    -   id: check-added-large-files
+-   repo: https://github.com/psf/black
+    rev: 20.8b1 # Replace by any tag/version: https://github.com/psf/black/tags
+    hooks:
+    -   id: black
+        language_version: python3 # Should be a command that runs python3.6+
+-   repo: https://github.com/pre-commit/mirrors-mypy
+    rev: v0.720
+    hooks:
+    -   id: mypy
+        exclude: ^(docs/|example-plugin/|tests/fixtures)
+-   repo: local
+    hooks:
+      - id: pytest
+        name: Check pytest unit tests pass
+        entry: poetry run duty test
+        pass_filenames: false
+        language: system
+        types: [python]

project/README.md.jinja

@@ -1,8 +1,7 @@
 # [[ project_name ]]
 
 [% if repository_provider == "gitlab.com" %][![pipeline status](https://gitlab.com/[[ repository_namespace ]]/[[ repository_name ]]/badges/master/pipeline.svg)](https://gitlab.com/[[ repository_namespace ]]/[[ repository_name ]]/pipelines)
-[![coverage report](https://gitlab.com/[[ repository_namespace ]]/[[ repository_name ]]/badges/master/coverage.svg)](https://gitlab.com/[[ repository_namespace ]]/[[ repository_name ]]/commits/master)[% elif repository_provider == "github.com" %][![ci](https://github.com/[[ repository_namespace ]]/[[ repository_name ]]/workflows/ci/badge.svg)](https://github.com/[[ repository_namespace ]]/[[ repository_name ]]/actions?query=workflow%3Aci)[% endif %]
-[![documentation](https://img.shields.io/badge/docs-mkdocs%20material-blue.svg?style=flat)](https://[[ repository_namespace ]].github.io/[[ repository_name ]]/)
+[![coverage report](https://gitlab.com/[[ repository_namespace ]]/[[ repository_name ]]/badges/master/coverage.svg)](https://gitlab.com/[[ repository_namespace ]]/[[ repository_name ]]/commits/master)[![documentation](https://img.shields.io/badge/docs-mkdocs%20material-blue.svg?style=flat)](https://[[ repository_namespace ]].gitlab.io/[[ repository_name ]]/)[% elif repository_provider == "github.com" %][![ci](https://github.com/[[ repository_namespace ]]/[[ repository_name ]]/workflows/ci/badge.svg)](https://github.com/[[ repository_namespace ]]/[[ repository_name ]]/actions?query=workflow%3Aci)[![documentation](https://img.shields.io/badge/docs-mkdocs%20material-blue.svg?style=flat)](https://[[ repository_namespace ]].github.io/[[ repository_name ]]/)[% endif %]
 [![pypi version](https://img.shields.io/pypi/v/[[ repository_name ]].svg)](https://pypi.org/project/[[ repository_name ]]/)
 [![gitter](https://badges.gitter.im/join%20chat.svg)](https://gitter.im/[[ repository_name ]]/community)
 

project/config/pytest.ini

@@ -13,3 +13,5 @@ addopts =
   --cov
   --cov-append
   --cov-config config/coverage.ini
+testpaths =
+    tests

project/duties.py

@@ -221,6 +221,7 @@ def clean(ctx):
     ctx.run("rm -rf .coverage*")
     ctx.run("rm -rf .mypy_cache")
     ctx.run("rm -rf .pytest_cache")
+    ctx.run("rm -rf tests/.pytest_cache")
     ctx.run("rm -rf build")
     ctx.run("rm -rf dist")
     ctx.run("rm -rf pip-wheel-metadata")
@@ -263,6 +264,18 @@ def docs_deploy(ctx):
     """
     ctx.run("mkdocs gh-deploy", title="Deploying documentation")
 
+
+@duty
+def docs_deploy_gitlab(ctx):
+    """
+    Deploy the documentation on Gitlab Pages.
+
+    Arguments:
+        ctx: The context instance (passed automatically).
+    """
+    ctx.run("mkdocs build -s --site-dir public", title="Deploying documentation")
+
+
 
 @duty
 def format(ctx):  # noqa: W0622 (we don't mind shadowing the format builtin)
@@ -299,7 +312,8 @@ def release(ctx, version):
         ctx.run("git push --tags", title="Pushing tags", pty=False)
         ctx.run("poetry build", title="Building dist/wheel", pty=PTY)
         ctx.run("poetry publish", title="Publishing version", pty=PTY)
-        ctx.run("mkdocs gh-deploy", title="Deploying documentation", pty=PTY)
+        # building docs with the site_dir specified in mkdocs.yml ensures the proper path 'site' or 'public' is used and this will work
+        ctx.run("mkdocs build", title="Deploying documentation")
 
 
 @duty(silent=True)