Implemented documentation and API docs build via Sphinx and fixed doc…

…strings

.github/workflows/tests.yml

@@ -17,7 +17,7 @@ jobs:
     test-venv:
         strategy:
             matrix:
-                python-version: ['3.11', '3.10', '3.9', '3.8', 'pypy3.9', 'pypy3.8']
+                python-version: ['3.x', '3.10', '3.9', '3.8', 'pypy3.9', 'pypy3.8']
                 platform: [ubuntu, windows]
                 architecture: [x64, x86]
                 exclude:
@@ -215,7 +215,7 @@ jobs:
             - name: Set up latest Python
               uses: actions/setup-python@v4
               with:
-                  python-version: '3.11'
+                  python-version: '3.x'
 
             - name: Install package in editable mode (temporarily) and its dependencies
               run: ./venv.sh install-venv
@@ -237,6 +237,15 @@ jobs:
                   name: curldl-dist
                   path: dist/
 
+            - name: Build Sphinx documentation
+              run: misc/scripts/run-sphinx.sh
+
+            - name: Upload documentation artifact
+              uses: actions/upload-artifact@v3
+              with:
+                  name: curldl-docs
+                  path: build/docs/
+
 
     publish-test-pypi:
         name: Publish to PyPI and GitHub on version tag

.gitignore

@@ -16,3 +16,4 @@ __pycache__/
 /build/
 /dist/
 /src/*.egg-info/
+/docs/api/*.rst

.readthedocs.yaml

@@ -0,0 +1,22 @@
+version: 2
+
+build:
+    os: ubuntu-22.04
+    apt_packages:
+        - libcurl4-openssl-dev
+    tools:
+        python: "3"
+
+sphinx:
+    configuration: docs/conf.py
+    fail_on_warning: true
+
+formats:
+    - htmlzip
+
+python:
+     install:
+         - method: pip
+           path: .
+           extra_requirements:
+               - doc

README.md

@@ -5,7 +5,7 @@
 
 # Introduction
 
-The __curldl__ Python module safely and reliably downloads files with [PycURL](https://pycurl.io/), which in turn is a wrapper for [libcurl](https://curl.se/libcurl/) file transfer library. The purpose of __curldl__ is providing a straightforward API for downloading files with the following features:
+The __curldl__ Python module safely and reliably downloads files with [PycURL](http://pycurl.io/), which in turn is a wrapper for [libcurl](https://curl.se/libcurl/) file transfer library. The purpose of __curldl__ is providing a straightforward API for downloading files with the following features:
 
 * Multi-protocol support: protocol support is delegated to [curl](https://curl.se/) in as protocol-neutral way as possible. This means that there is no reliance on HTTP-specific header and statuses, for example. If a feature like download resuming and _if-modified-since_ condition is supported by the underlying protocol, it can be used by _curldl_.
 * If a partial download is abandoned, most chances are that it may be resumed later (supported for HTTP(S), FTP(S) and FILE protocols). A `.part` extension is added to the partial download file, and it is renamed to the target file name once the download completes.
@@ -29,7 +29,7 @@ If you encounter a build failure during installation of _pycurl_ dependency, the
 * On Windows, install an unofficial _pycurl_ build since official builds are not available at the moment — e.g., by [Christoph Gohlke](https://www.lfd.uci.edu/~gohlke/pythonlibs/#pycurl), or use _Conda_ (see below).
 * On Windows and macOS, use _Conda_ or _Miniconda_ with [conda-forge](https://conda-forge.org/) channel. For instance, see runtime dependencies in the following [test environment](https://github.com/noexec/curldl/blob/develop/misc/conda/test-environment.yml).
 
-Overall, _curldl_ is expected to have no issues in any environment with Python 3.8+ (CPython or PyPy) — see Testing section below.
+Overall, _curldl_ is expected to have no issues in any environment with Python 3.8+ (CPython or PyPy) — see [Testing](#testing) section below.
 
 
 # Usage
@@ -176,6 +176,12 @@ In order to run tests locally with Python interpreter available in the system, i
 
 `venv.sh` is a convenience _venv_ wrapper that also enables some additional Python checks; you can simply activate the _venv_ environment instead. Testing with _Conda_ is possible as well — see the [CI/CD pipeline execution](https://github.com/noexec/curldl/actions) for details.
 
+
+# Changelog
+
+See [Changelog](https://github.com/noexec/curldl/blob/develop/docs/CHANGELOG.md) file for a summary of changes in each release.
+
+
 # License
 
 This project is released under the [GNU LGPL License Version 3](https://github.com/noexec/curldl/blob/develop/LICENSE.md) or any later version.

docs/conf.py

@@ -0,0 +1,85 @@
+import subprocess  # nosec
+from importlib import metadata
+
+
+project = 'curldl'
+copyright = '2023, Michael Orlov'
+
+release = metadata.version(project)
+version = '.'.join(release.split('.')[:2])
+release = '.'.join(release.split('.')[:3])
+
+nitpicky = True
+
+extensions = [
+    'myst_parser',
+    'sphinx.ext.autodoc',
+    'sphinx.ext.autosectionlabel',
+    'sphinx.ext.intersphinx',
+    'sphinx.ext.viewcode',
+    'sphinx_copybutton'
+]
+
+source_suffix = {
+    '.md': 'markdown',
+    '.rst': 'restructuredtext'
+}
+
+exclude_patterns = [
+    'CHANGELOG.md',
+    'changelog.d/**'
+]
+
+suppress_warnings = [
+    'autosectionlabel.*'
+]
+
+myst_enable_extensions = [
+    'colon_fence',
+    'deflist',
+    'dollarmath',
+    'fieldlist',
+    'linkify',
+    'replacements',
+    'smartquotes',
+    'strikethrough'
+]
+
+myst_linkify_fuzzy_links = False
+
+intersphinx_mapping = {
+    'python': ('https://docs.python.org/3/', None),
+    'pycurl': ('http://pycurl.io/docs/latest/', None),
+    'tenacity': ('https://tenacity.readthedocs.io/en/latest/', None)
+}
+
+nitpick_ignore = [
+    ('py:class', 'tqdm'),
+    ('py:exc', 'argparse.ArgumentError'),
+    ('py:exc', 'metadata.PackageNotFoundError'),
+    ('py:class', 'pycurl.error'),
+    ('py:exc', 'pycurl.error')
+]
+
+html_theme = 'furo'
+
+html_theme_options = {
+    'navigation_with_keys': True,
+    'top_of_page_button': None
+}
+
+autoclass_content = 'both'
+
+autodoc_default_options = {
+    'members': True,
+    'private-members': True,
+    'undoc-members': True,
+    'special-members': True,
+    'member-order': 'bysource'
+}
+
+autodoc_typehints = "both"
+autodoc_typehints_description_target = "documented"
+
+subprocess.run(f'sphinx-apidoc -feM -o api ../src/curldl'.split(),  # nosec
+               check=True, text=True, encoding='ascii')

docs/index.md

@@ -0,0 +1,11 @@
+```{include} readme.md
+:end-before: "# Installation"
+```
+
+
+```{toctree}
+:hidden:
+
+readme.md
+API Reference <api/modules.rst>
+```

docs/readme.md

@@ -0,0 +1 @@
+../README.md

misc/scripts/run-bandit.sh

@@ -5,7 +5,7 @@ script_dir=${0%/*}
 [ "${script_dir}" != "$0" ] || script_dir=.
 
 project_dir="${script_dir}/../.."
-code_roots="src tests"
+code_roots="src tests docs"
 
 cd "${project_dir}"
 bandit -c pyproject.toml -r ${code_roots} "$@"

misc/scripts/run-sphinx.sh

@@ -0,0 +1,13 @@
+#!/bin/sh
+set -e
+
+script_dir=${0%/*}
+[ "${script_dir}" != "$0" ] || script_dir=.
+
+project_dir="${script_dir}/../.."
+docs_root="docs"
+build_docs_root="build/docs"
+doctrees_root="build/tests/doctrees"
+
+cd "${project_dir}"
+sphinx-build -vW -d "${doctrees_root}" "${docs_root}" "${build_docs_root}" "$@"

pyproject.toml

@@ -62,12 +62,18 @@ dev = [
     "pipdeptree",           # show packages dependency tree
     "pip-autoremove",       # remove unused package dependencies
     "pip-review",           # upgrade outdated packages
-    "sphinx",               # documentation generator
     "towncrier",            # producing changelogs
     "twine",                # PyPI package publishing
     "wheel"                 # build system, support PEP 518 installs
 ]
 
+doc = [
+    "myst-parser[linkify]", # Markdown MyST parser
+    "sphinx",               # documentation generator
+    "sphinx-copybutton",    # "copy" button for code blocks
+    "furo"                  # Sphinx HTML theme
+]
+
 [project.urls]
 "Homepage" = "https://github.com/noexec/curldl"
 "Documentation" = "https://pypi.org/project/curldl/"

src/curldl/cli.py

@@ -25,6 +25,7 @@ def __init__(self) -> None:
     def _configure_logger(args: argparse.Namespace) -> None:
         """Configure logger according to command-line arguments.
         Specifying `verbose` argument raises the log level to `debug`.
+
         :param args: command-line arguments
         """
         debug_log_level = 'debug'
@@ -40,7 +41,8 @@ def _configure_logger(args: argparse.Namespace) -> None:
 
     @classmethod
     def _parse_arguments(cls) -> argparse.Namespace:
-        """Parse command-line arguments
+        """Parse command-line arguments.
+
         :return: arguments after configuring the logger and possibly inferring other arguments
         """
         parser = argparse.ArgumentParser(prog=__package__, formatter_class=argparse.ArgumentDefaultsHelpFormatter)
@@ -74,11 +76,12 @@ def _parse_arguments(cls) -> argparse.Namespace:
 
     @classmethod
     def _infer_arguments(cls, output_arg: argparse.Action, args: argparse.Namespace) -> argparse.Namespace:
-        """Infer missing arguments
+        """Infer missing arguments.
+
         :param output_arg: `output` argument to infer
         :param args: arguments to extend
         :return: input arguments after inferring missing ones
-        :raises argparse.ArgumentError: multiple URLs are specified with ``output`` argument
+        :raises ``argparse.ArgumentError``: multiple URLs are specified with ``output`` argument
         """
         if not args.output:
             args.output = [os.path.basename(urllib.parse.unquote(urllib.parse.urlparse(url).path)) for url in args.url]
@@ -90,7 +93,8 @@ def _infer_arguments(cls, output_arg: argparse.Action, args: argparse.Namespace)
         return args
 
     def main(self) -> object:
-        """Command-line program entry point
+        """Command-line program entry point.
+
         :return: program exit status
         """
         dl = Curldl(self.args.basedir, progress=self.args.progress, verbose=self.args.verbose)
@@ -101,7 +105,8 @@ def main(self) -> object:
 
     @staticmethod
     def _get_package_version() -> str:
-        """Retrieve package version from metadata, raising error for uninstalled development sources
+        """Retrieve package version from metadata, raising error for uninstalled development sources.
+
         :return: package version string
         :raises metadata.PackageNotFoundError: version is not available, e.g. when package is not installed
         """
@@ -113,7 +118,8 @@ def _get_package_version() -> str:
 
 
 def main() -> object:
-    """Command-line static entry point, suitable for install-time script generation
+    """Command-line static entry point, suitable for install-time script generation.
+
     :return: program exit status
     """
     return CommandLine().main()