Improve docs deployment #87
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 11 commits
November 11, 2020 23:07
j-ittner
approved these changes
Nov 13, 2020
This was referenced Nov 17, 2020
konst-int-i
added a commit
that referenced
this pull request
Jan 11, 2021
* mark /dist as excluded to prevent scanning * added __all__ statement * added __all__ statement * added __all__ statement * added __all__ statement * added __all__ statement * removed __all__ statement * added __all__ statement * added __all__ statement * removed __all__ statement * removed __all__ statement * removed __all__ statement * added __all__ statement * removed __all__ statement * removed __all__ statement * removed __all__ statement * added __all__ statement * removed __all__ statement * added __all__ statement * removed __all__ statement * added __all__ statement * removed __all__ statement * removed __all__ statement * start of pytools sphinx doc * start of pytools sphinx doc * start of pytools sphinx doc * start of pytools sphinx doc * CONDA: update conda build path to replace spaces * CI/CD: Setup CI with azure-pipelines.yml * CI/CD: specify py_modules for pip installation * TESTS: add pytest.ini and setup.cfg * TESTS: add coverage.xml to pytest.ini * CI/CD: add pytest coverage tasks to azure-pipelines.yml * CI/CD: change default paths for code coverage tools * CI/CD: add git credentials to run make package * CI/CD: add git username as library VAR * CI/CD: define variables in step * CI/CD: fix calling of env variables * CI/CD: reference variable group * CI/CD: set username * CI/CD: fix typo * CI/CD: update conda build to take local path * CI/CD: run make package for pytools * CI/CD: run make package for pytools * CI/CD: run make package for pytools * CI/CD: export PYTHONPATH * CI/CD: export PYTHONPATH * CI/CD: update conda meta.yaml to run coverage report * CI/CD: run tests in azure-pipelines.yml exclusively * CI/CD: clean up azure pipeline * DOCS: remove --implicit-namespace flag from Makefile * DOCS: add BCG copyright * CI/CD: add dependent steps into azure-pipelines.yml * CI/CD: add fresh environment for conda build * CI/CD: set environment variables from strategy matrix * CI/CD: do not ask for confirmation on env build * CI/CD: update conda-build dependency * CI/CD: update conda-build dependency * CI/CD: update minimum FACET_V_PYTHON_BUILD to v3.6 * updates to make * minor updates to conf * changed readme to rst * updates to doc layout * new reamdme in rst format * api doc layout * contribution guide layout * faq layout * getting started layout * image for readme * revisions to formatting * adding API folder to api doc opts * BUILD: change to readme.rst in setup.py * removing uneccesary text * removing m2r * adding package blurb * DOCS: adds extensive contribution guide for sphinx docs * ENV: add environment.yml for pytools * ENV: add dev-setup.sh for pytools * ENV: update contribution_guide.rst * rename gamma-viz project to pytools; remove pytools/.idea/gamma-viz.iml * DOCS: remove redundant sphinx setup instructions * ENV: activate env in shell script * This commit will make minor corrections and revisions to contribution guidelines * ENV: updates shebang line to use /sh instead of /bash * CI/CD: update repositories to checkout * wrap text in a docstring * return NotImplemented for unsupported Python operators (#18) * improve number formatting in "matrix%" style to avoid very small numbers * use GAMMA FACET colours as the default colour scheme for matplot charts (#19) * use GAMMA FACET colours as the default colour scheme for matplot charts * API: improve how AllTracker determines the symbols it expects in __all__ * DATA: create new module pytools.data and move sim_data() here * DATA: remove old module pytools.data.simulation from sphinx api docs * comprehensive update & streamlining of build mechanism for sphinx documentation (#23) * SPHINX: deactivate option show-inheritance; we generate this ourselves * SPHINX: update custom code to generate cross-references and list bases * SPHINX: update configuration * SPHINX: move custom autodoc processors from conf.py to pytools library * SPHINX: push __all__ statements down from __init__.py to private modules * DOC: fix pytools.expression docstrings * SPHINX: fix __all__ statement in _sphinx.py * SPHINX: use new method connect() to register sphinx event callbacks * SPHINX: stop auto=generating API doc in the sphinx Makefiles * SPHINX: mark sphinx/build as excluded in PyCharm * make minor updates to inline comments * SPHINX: add module pytools.sphinx to sphinx api docs * update pytools.sphinx type hints and docstrings * add intersphinx link to sphinx * include typing_inspect package from conda (not pypi) to fix conda build * Docs/use py script for docs (#25) * replaced with make.py * replaced with make.py * added python script for sphinx doc build * revised to point to make.py * simplified make.py * deactivate autodoc typehints option always_document_param_types (#27) * Add a logo file for the ReadMe to import (#26) * fine-tune the standard color gradient for a more "linear" progression (#28) * push SphinxCallback.process() and __call__() down to AutodocProcessor (#29) * reduce scope of tests in azure pipeline for develop branch (#31) * reduce scope of tests in azure pipeline for develop branch * update environment.yml * update package versions in conda build recipe * Share code for Sphinx make.py and conf.py across all repos (#30) This commit: * Generalized make.py into a universal makefile to build documentation for pytools and other repos. * Moved majority of sphinx config to shared conf_base file that also calls newly added custom css and js files in pytools and added "transform_notebook" script. * bugfix: get canonical path of __file__ before performing path operations (#32) * change make.py into a universal makefile, not limited to pytools * use the current working directory as the make.py working directory * add the `transform_notebook` script to the pytools sphinx scripts * move bulk of sphinx config to shared config_base in pytools * ensure python path is set up to access `source.scripts` module * add custom css and js files from within conf_base script * generalize the description of the html command in make.py * replace `path.dirname(__file__)` with `path.dirname(path.realpath(__file__))` * update conditional build logic depending on target branch and scheduling (#33) * replace wildcard imports with explicit import statements; upgrade black to v20.8b1 (#36) * replace wildcard imports with explicit import statements * upgrade black to v20.8b1 * Add code checks to the CI pipeline: isort, black, flake (#34) * add isort, black, and flake8 to pipeline * add isort and flake8 to pre-commit hooks * reformat code to conform with new code inspections Co-authored-by: Jan Ittner <ittner.jan@bcg.com> * Update CI pipeline configuration (avoid black/flake8 conflicts; report failed unit test) (#37) * update errors ignored by flake8 to avoid conflicts with black * force pipeline to stop when a unit test fails * Add full Sphinx configuration to global scope in method set_config() (#38) * add AllTracker to remaining pytools modules (#39) * add AllTracker to remaining pytools modules * update docstring of AllTracker class * complete __all__ for viz.distribution and viz.matrix; add unit test (#40) * Update structure and design of Sphinx documentation (#41) * use recursive autosummary to generate API reference (new in Sphinx 3.1) * update logos * update CSS styling with GAMMA/BCG CI * move common content of README.rst and getting_started.rst to readme.rst * add API index to TOC tree * update headings on index page * declare sphinx/source a Python source folder * Restore README.rst (#42) * remove README.rst * move full "Getting started" page to README.rst * restore header on Getting Started page * fix GAMMA PYTOOLS logo (lowest row of pixels was cropped) * update the command mechanism in make.py (#44) * update sphinx build mechanism for increased portability across repos (#46) * update develop environment to ensure error-free documentation build * update make.py for increased portability with other Gamma repos * move conda_base and make to new directory sphinx/base * delete apidoc folder upon make clean * align folder exclusions with .gitignore * update/fix LICENSE file * move transform_notebook.py to base folder; fix import to make_base.py (#47) * move transform_notebook.py to base folder; fix import to make_base.py * update nbsphinx to v0.7 for compatibility with nbconvert 6 * Fix path to base scripts for sphinx build (#48) * Strengthen pre-commit checks (#49) * fix target path for css and js files in Sphinx html documentation (#50) * Update flake8 pre-commit hook to use local configs (#51) * Strengthen pre-commit checks * Update flake8 pre-commit to use local configs Co-authored-by: Jan Ittner <ittner.jan@bcg.com> * add css config to base for personnel photos (#53) * Link Python/Conda package version strings (#52) * add make.py to eventually replace Makefile * clarify script purpose in docstring * set chmod+x on make.py * add a makedirs() for the dist/ folder * make file:/ URI using pathlib for Windows support * add & reference make.py in pytools/condabuild * remove Makefile and use make.py in pipeline * conda build: move make.py and recipe * CI/CD: correct path to make.py * CI/CD: add meta.yaml in new path * link version strings across setup.py, __init__ & conda * rename conda-build to condabuild in azure-pipelines.yml (#54) * correctly detect docstrings in decorated methods (#57) * Fix/improve docstrings for class Expression (#56) * Move viz base classes to `*.base` submodule; move viz utilities to `pytools.viz.util` (#59) This commit * moves linkage tree base classes to pytools.viz.dendrogram.base * moves dendrogram style base classes to pytools.viz.dendrogram.base * moves colour constants and PercentageFormatter to pytools.viz.util * defines Facet color scheme in module pytools.viz.colors * add gitbadges and initial FAQs (#55) This commit * adds placeholders for git badges * adds initial FAQ section * revision to colour of hyperlinks to make them standout better * Detail a warning message in inheritdoc decorator (#61) This commit * clarifies a warning that is issued when the inheritdoc decorator does not find the match string anywhere in the class * fixes some match strings based on where the warning was emitted * re-enable custom bases and generic types rubrics for class documentation (#60) * Add FacetTeam email in the FAQ (#63) * Add sphinx/base to the Python source path (#64) * CI/CD: define trigger and pr sections for pipeline (#66) * Update PyCharm configuration, incl. newly introduced PyCharm features (#65) * Prepare versioned docs builds and expand HTML/JS client with selector (#67) * sphinx build: add javascript file that will contain docs versions * ignore changes to versions.js * sphinx make: delete .gitignore – does not work for already tracked files * sphinx make: add target that pulls existing versions and stores in JS * sphinx make: improve by descendingly sorting versions from new to old * Sphinx make: rename versions const to DOCS_VERSIONS * sphinx make: make_base take version from setup.py and copy folder * expand client side javascript for Docs version management * sphinx make: Clean dependency not needed for FetchPkgVersions * environment.yml: add toml dependency * docs: bugfix fetching currently active version^ * sphinx-docs: correct a JS bug in gamma.js * Remove "The" from BCG in the copyright (#70) * Correct signature of ParallelizableMixin._delayed (#68) * Refactor build ecosystem (#69) * sphinx build: add javascript file that will contain docs versions * ignore changes to versions.js * sphinx make: delete .gitignore – does not work for already tracked files * sphinx make: add target that pulls existing versions and stores in JS * sphinx make: improve by descendingly sorting versions from new to old * sphinx make: rename versions const to DOCS_VERSIONS * sphinx make: make_base take version from setup.py and copy folder * expand client side javascript for Docs version management * sphinx make: Clean dependency not needed for FetchPkgVersions * environment.yml: add toml dependency * docs: bugfix fetching currently active version^ * sphinx-docs: correct a JS bug in gamma.js * README.rst: set 'https://' in TBD links for build to accept rst file * build: add minimal tox.ini for PyPi package matrix tests * build: add pyproject.toml for flit based PyPi build * build: move make.py to toplevel of repo for common build tasks * build: completely redo make.py to support tox build and inject deps. * build: adapt condabuild/meta.yaml for new pkg deps. injection * build: add tox & flit dependency to environment.yml * pycharm: add .tox as excluded folder to avoid scanning * build: use new build toolchain in azure-pipelines.yml * build: bugfix use renamed detect_build_config_changes * make.py: remove trailing whitespace * build: install tox, flit, toml * build: call make.py from devops python * build: print python executable for debug * build: use conda bash shell hook if conda build * build: clarify job display name * build: switch between vanilla and Conda Python * build: use flit build & install in meta.yaml * build: add sdist exclusions to pyproject.toml * build: pyproject.toml add dist-name * build: use pip v20 and flit in conda build env (meta.yaml) * build: remove obsolete setup.py * build: meta.yaml retrieve pkg build version from env variable * build: make.py retrieve pkg version (without setup.py) and expose * CI: consolidate flake8.ini into tox.ini * CI: consolidate setup.cfg into tox.ini * CI: consolidate .isort.cfg into tox.ini * CI: consolidate pytest.ini into tox.ini * sphinx build: make_base.py improve path constants; add FACET_PROJECT * sphinx build: make_base.py remove setup.py dependency for __version__ * sphinx build: azure-pipelines.yml set correct flake config * build: make.py bugfix: importlib.util import error * build: update testing & docs requirements in pyproject.toml * build: update testing & docs requirements in pyproject.toml * pre-commit hook: update flake8 config to use tox.ini * pyproject.toml: correct unconstrained min. deps; set Py3.7 as max * pyproject.toml: set 3.8 as maximum Python version * Prepare support for local PyPI cross package dependencies (#71) * build: tox.ini - define distshare, so that builds are kept * build: tox.ini - define distshare as dist/tox to allow isolated clean * build: make.py: improve clean, create local index for PyPi packages * build bugfix bash env variables can't use dash change to underscore (#72) * clean up pytools.viz code and docstrings (#73) * update ReadMe sections (#74) * CI: Add quick build test for PyPI (#75) * simplify the logo banners on documentation landing pages (#76) * Bugfix/add facet path for windows (#77) * DOC: Render and publish on GitHub Pages * CI/CD: Fix folders * DOC: Fix paths for the case of no "pytools" parent directory * CI/CD: Change default working dir "s" to "pytools" for sphinx/make.py to work as expected * CI/CD: Removed duplicate pytools root in paths, already contained in working dir * CI/CD: Check that docs build is carried to next task * DOC: Removed docs from .gitignore to allow publishing to a branch * CI/CD: Add verbosity and remove pushing until credentials are in place * CI/CD: Add only contents in docs folder * CI/CD: Add access to deploy key and publish docs * update channel for conda install (#78) * CI/CD: Force use of SSH authentication * CI/CD: Try different enforcing method for SSH * CI/CD: Add full docs folder, including .nojekyll * CI/CD: Fix interference of new docs build with old ones * CI/CD: Remove .buildinfo to fix GitHub Pages build * CI/CD: Move latest rendering of docs to 'latest' version folder * CI/CD: Move docs version handling to Python * CI/CD: Let publishing task fail when one command fails * DOCS: Change deployment branch to be the more canonically named 'gh-pages' * CI/CD: Reinstantiate triggers * DOCS: Revert from auto-publishing branch name 'gh-pages' to 'github-pages' for more control * CI/CD: Fix docs version updates by checking for existing version history * CI/CD: Use pytools conda env for updating docs versions * CI/CD: Add missing conda hook * CI/CD: Reinstate all pipeline stages * DOCS: Fix formatting * CI/CD: Fix docs updates when docs already exist * CI/CD: Store SSH public key and known hosts entry as variables * CI/CD: Fix variable group syntax * CI/CD: Update docs version.js to latest one for all older versions * CI/CD: Rerun pipeline with updated make_base.py on branch 'github-pages' * CI/CD: Only run docs stage for releases * Correct path to static_base in make_base.py (#79) * add flow to known_projects (#83) * add flow to _version in get_package_version (#85) * add flow to get_package_version (#86) * Release pipeline (#84) * CI/CD: Store conda/tox builds as artifacts * CI/CD: Reformat comments * CI/CD: Add initial version of release stage * CI/CD: Fix trigger to release only on master branch, excluding PRs to master * CI/CD: Reinstate restrictive build test condition * CI/CD: Automatically detect pre-release * CI/CD: Add dependency for release * CI/CD: Remove wrong dependency of docs on release * make pytools italic, minor tidy up to readme and index removal (#88) * update pypi and conda package build metadata (#82) * Improve docs deployment (#87) * CI/CD: Prepare docs deployment from pipeline-triggering branch * CI/CD: Install 'tree' for more transparency in Azure logs * CI/CD: Add empty docs folder in case no docs have been deployed before * CI/CD: Switch to original branch to run docs update script * CI/CD: Make sure to be in root directory with .git folder * CI/CD: Use actual branch name instead of 'pull/xx/merge' * Warn on spaces in build path and allow override using ENV variable (#80) * Bugfix: Repair docs update when old version is present (#90) * CI/CD: Fix git checkout issue by removing docs folder * CI/CD: Reinstate restrictive condition for deploying new docs version * DOCS: Create notebook dir if it does not exist (#92) * add package badges (#91) * add trove classifiers to pypi metadata (#89) * add azure badge links and update conda install instructions (#93) * remove the notebook transform and copying to a nb folder in root dir (#94) * Improve release pipeline (#95) * modify readme prior to integrating with getting started documentation in Sphinx build (#96) * update doc guidelines (#97) * enhance process to build Sphinx documentation (#98) * Various small improvements and fixes of pytools code; completion of docstrings (#99) * CI/CD: Use bcg_gamma channel for conda upgrade instructions in release notes (#100) * update documentation of package pytools.expression, along with several minor API improvements (#101) * add API landing page introduction via custom module template (#102) * add API landing rst to documentation guide (#103) * update development guide (#105) * move selected expression classes to submodules base, atomic, and composite (#104) * fix docstrings (#106) * remove trailing backslashes in docstrings: not needed in canonical ReST * unify case and formatting of *matplotlib* * remove class ImportGroup and associated subclasses/API (#108) * remove trailing backslashes in docstrings: not needed in canonical ReST (#107) * remove trailing backslashes in docstrings: not needed in canonical ReST * unify case and formatting of *matplotlib* * replace tabs with spaces in contribution guide (#112) * minor code tweaks to eliminate IDE warnings (#111) * bugfix: exception message in pytools.api.validate_type (#110) * increase reliability of converting ForwardRef types to Sphinx xrefs (#113) * improve rendering of percentages > 100% in PercentageFormatter (#114) * introduce class ColorScheme to module pytools.viz (#109) * make DrawStyle methods public: start_drawing(), finalize_drawing() (#115) * Improve OO implementation of named drawing styles (#117) * replace 3rd party docstrings with a reference to the 3rd party module (#118) * add issue templates (#116) * fix references to DrawingStyle.…_drawing methods (#119) * updates to intersphinx and minor text edits (#120) * make process for doc build more visible (#121) * rename type variables for methods' "self" arguments to T_Self (#122) * further doc string tidy up (#123) * update guide link (#124) * set the default range of the PercentageMatrix colorbar to [0.0,1.0] (#126) * remove badge (#128) * Improve handling of 3rd party module references in Sphinx documentation (#129) * refactor ECDF and XYSeries to not base NamedTuple for clearer API & docs (#130) * fix typos in cross-references (#131) * restore compatibility with Python 3.6 (#132) * complete a comment (#133) * Release/1.0.1 (#134) * set the default range of the PercentageMatrix colorbar to [0.0,1.0] * CI/CD: Fix master_or_release variable by changing to runtime processing * CI/CD: Fix master_or_release variable by inlining to simplify YAML parsing * CI/CD: Remove double pipeline runs for release branch (only PR to master should trigger) * update doc path for conda build * update python unconstr dependency * remove ,4 for python unconstr depen on DevOps * increase minimum python to 3.7 * adjust gitbadge * remove additional single quote * edit summary * update classifiers * add full image path * restore link * restore min dep python 3.6 * update git badge python * revise min python dep * update details and links * restore LICENSE * improve formatting for pypi page * DOCS: update target of sphinx badge * descr for pypi * restore original * fix line * update doc link in badge * DOCS: change latex extension to mathjax * DOCS: change latex extension to mathjax * bump initial release version Co-authored-by: Jan Ittner <ittner.jan@bcg.com> Co-authored-by: Kennedy Ricardo <Kennedy.Ricardo@bcg.com> Co-authored-by: Jason Bentley <Bentley.Jason@bcg.com> Co-authored-by: Konstantin Hemker <hemker.konstantin@bcg.com> * update doc string (#135) * Initialize release Co-authored-by: Jörg Schneider <46053259+joerg-schneider@users.noreply.github.com> Co-authored-by: Jörg Schneider <schneider.joerg@bcg.com> Co-authored-by: Konstantin Hemker <hemker.konstantin@bcg.com> Co-authored-by: Konstantin Hemker <33329141+konst-int-i@users.noreply.github.com> Co-authored-by: Jan Ittner <ittner.jan@bcg.com> Co-authored-by: Jan Ittner <GitHub.com@ittner.de> Co-authored-by: mgelsm <malo.grisard@pm.me> Co-authored-by: mgelsm <grisard.malo@bcg.com> Co-authored-by: sithankanna <sithankanna@gmail.com> Co-authored-by: Ricardo Kennedy <kennedy.ricardo@bcg.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR improves the docs deployment:
github-pagesreduced to only containdocsfolder and no code (besidesazure-pipelines.ymlwith one-liner to disable CI for the branch)python sphinx/make.py prepare_docs_deploymentscript run from branch that triggers the deployment, instead of using code ongithub-pagesbranch that would have to be manually updated regularlyAdditionally, it fixes the following:
branchNameadjusted so that for PRs it's the source (current) branch, rather than the target branch (e.g.,developfor this PR)