Import pinefarm from runcards repo

.github/CONTRIBUTING.md

@@ -0,0 +1,12 @@
+## Dependencies
+
+### More on tools
+
+The `runcardsrunner` package is managed by `poetry`. This has been chosen to
+automate a series of tasks, and to grant reproducible installations maintaining
+a `poetry.lock` file.
+
+It might happen (frequently) that `poetry` complains about version
+dependencies, if inconsistent. This is not `poetry`'s fault, and it's a useful
+warning that will prevent to just break when installed by an user with a
+different environment.

.github/workflows/pypi.yml

@@ -0,0 +1,12 @@
+name: deploy
+
+on:
+  push:
+    tags:
+      - "*"
+
+jobs:
+  publish:
+    uses: N3PDF/workflows/.github/workflows/python-poetry-pypi.yml@v2
+    secrets:
+      PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}

.github/workflows/unittests.yml

@@ -0,0 +1,13 @@
+name: tests
+
+on: push
+
+jobs:
+  test:
+    strategy:
+      matrix:
+        python-version: ["3.8", "3.9", "3.10"]
+
+    uses: N3PDF/workflows/.github/workflows/python-poetry-tests.yml@v2
+    with:
+      python-version: ${{ matrix.python-version }}

README.md

@@ -0,0 +1,33 @@
+# Pinefarm
+
+Generate the corresponding PineAPPL grids.
+
+## Installation
+
+There is no released version currently.
+
+### Dev
+
+For development you need the following tools:
+
+- `poetry`, follow [installation
+  instructions](https://python-poetry.org/docs/#installation)
+- `poetry-dynamic-versioning`, used to manage the version (see
+  [repo](https://github.com/mtkennerly/poetry-dynamic-versioning))
+- `pre-commit`, to run maintenance hooks before commits (see
+  [instructions](https://pre-commit.com/#install))
+
+See [below](.github/CONTRIBUTING.md#non-python-dependencies) for a few more
+dependencies (already available on most systems).
+
+## Documentation
+
+The documentation is not deployed at the moment.
+In order to generate it install the project in development, and then do:
+
+```sh
+poetry shell
+cd docs
+make html
+make view
+```

docs/.gitignore

@@ -0,0 +1,6 @@
+# Ignore auto generated module references
+source/modules
+# ignore temporary build files
+_build/
+# Ignore generated sphinx-bibtex file
+source/bibtex.json

docs/Makefile

@@ -0,0 +1,35 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+PINEFARMOUT = $(SOURCEDIR)/modules
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+view:
+	$(BROWSER) $(BUILDDIR)/html/index.html
+
+server:
+	python3 -m http.server 8000 --bind 127.0.0.1 --directory build/html
+
+clean:
+	rm -rf build
+	rm -rf _build
+
+cleanall: clean
+	rm -rf $(PINEFARMOUT)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.http://sphinx-doc.org/
+	exit /b 1
+)
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

docs/source/cli.rst

@@ -0,0 +1,39 @@
+``pinefarm`` command line interface
+===================================
+
+This is not a replacement of the CLI help, you can get it with:
+
+.. code-block:: sh
+
+   pinefarm --help
+
+It is just a brief recap of the subcommands and their goal.
+
+``install``
+-----------
+
+Installs various programs, used to run pinecards.
+
+``run``
+-------
+
+It is the main command provided, and it runs the specified pinecard in the
+context of the selected theory.
+
+The output will be stored in a directory in the current path, with the name
+``<RUNCARD>-<YYYYMMDDhhmmss>``, where:
+
+- the first part is the name of the selected pinecard (that is also the name of
+  the folder in which all the files are stored)
+- the second part is the timestamp of the moment in which the command is issued
+
+``update``
+---------
+
+Update the metadata of the specified grid, with the content of the
+``metadata.txt`` file in the current version of the pinecard.
+
+``merge``
+---------
+
+Merge the specified grids' content into a new grid.

docs/source/conf.py

@@ -0,0 +1,152 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+# import os
+# import sys
+# sys.path.insert(0, os.path.abspath('.'))
+
+import os
+import pathlib
+import sys
+
+here = pathlib.Path(__file__).absolute().parent
+
+# -- Project information -----------------------------------------------------
+
+project = "pinefarm"
+copyright = "2021, the pinefarm team"
+author = "the pinefarm team"
+
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    "sphinx.ext.autodoc",
+    "sphinx.ext.doctest",
+    "sphinx.ext.intersphinx",
+    "sphinx.ext.todo",
+    "sphinx.ext.coverage",
+    "sphinx.ext.mathjax",
+    "sphinx.ext.ifconfig",
+    "sphinx.ext.viewcode",
+    "sphinx.ext.autosectionlabel",
+    "sphinx.ext.napoleon",
+    "sphinxcontrib.bibtex",
+    "sphinx.ext.graphviz",
+    "sphinx.ext.extlinks",
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ["_templates"]
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+#
+source_suffix = {
+    ".rst": "restructuredtext",
+    ".txt": "restructuredtext",
+}
+
+autosectionlabel_prefix_document = True
+# autosectionlabel_maxdepth = 10
+# Allow to embed rst syntax in  markdown files.
+enable_eval_rst = True
+
+# The master toctree document.
+master_doc = "index"
+bibtex_bibfiles = ["refs.bib"]
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ["shared/*"]
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = None
+
+# A string to be included at the beginning of all files
+shared = here / "shared"
+rst_prolog = "\n".join([open(x).read() for x in os.scandir(shared)])
+
+extlinks = {
+    "yadism": ("https://yadism.readthedocs.io/%s", "yadism"),
+    "banana": ("https://banana-hep.readthedocs.io/%s", "banana"),
+    "pineappl": ("https://docs.rs/pineappl/latest/pineappl/%s", "pineappl"),
+    "pineko": ("https://github.com/N3PDF/pineko/%s", "pineko"),
+}
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = "sphinx_rtd_theme"
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ["_static"]
+
+# -- Extension configuration -------------------------------------------------
+
+# -- Options for intersphinx extension ---------------------------------------
+
+# Example configuration for intersphinx: refer to the Python standard library.
+# Thanks https://github.com/bskinn/sphobjinv
+intersphinx_mapping = {
+    "python": ("https://docs.python.org/3/", None),
+    "scipy": ("https://docs.scipy.org/doc/scipy", None),
+    "numpy": ("https://numpy.org/doc/stable", None),
+    "pygit2": ("https://www.pygit2.org/", None),
+    "pandas": ("https://pandas.pydata.org/docs/", None),
+}
+# -- Options for todo extension ----------------------------------------------
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = True
+
+mathjax3_config = {
+    "tex": {
+        "macros": {
+            # fncs
+            # "atan": [r"\text{atan}", 0],
+            # "span": [r"\text{span}", 0],
+        }
+    }
+}
+
+
+# https://github.com/readthedocs/readthedocs.org/issues/1139#issuecomment-312626491
+def run_apidoc(_):
+    from sphinx.ext.apidoc import main  # pylint: disable=import-outside-toplevel
+
+    sys.path.append(str(here.parent))
+
+    # analyse 'pineappl'
+    docs_dest = here / "modules" / "pinefarm"
+    package = here.parents[1] / "pinefarm"
+    main(["--module-first", "-o", str(docs_dest), str(package)])
+    (docs_dest / "modules.rst").unlink()
+
+
+def setup(app):
+    app.connect("builder-inited", run_apidoc)

docs/source/external/index.rst

@@ -0,0 +1,41 @@
+External runners
+================
+
+The ``pinefarm run`` is mainly a uniform interface to some Monte Carlo (and non)
+generator that are able to produce PineAPPL grids.
+
+Internally the runners are managed through a class system, with a base class
+:class:`~pinefarm.external.interface.External`, that defines the basic
+steps and attributes, while implementing the common actions.
+
+Attributes:
+
+- ``name``: name of the dataset
+- ``theory``: identifier of the theory
+- ``pdf``: PDF used for the comparison
+- ``timestamp``, *optional*: the timestamp of the previous run, if rerunning an
+  already present grid
+
+Computed attributes:
+
+- ``dest``: folder used for all the output
+- ``source``: folder containing pinecard
+- ``grid``: path of the computed grid
+- ``gridtmp``: path used for auxiliary grid (removed at the end of the run)
+
+Steps:
+
+- :meth:`~pinefarm.external.interface.External.install`: further install
+  steps, needed for the runner (not needed if the runner available as a python
+  package on PyPI)
+- :meth:`~pinefarm.external.interface.External.run`: compute the actual
+  predictions
+- :meth:`~pinefarm.external.interface.External.generate_pineappl`: collect
+  predictions into a |pineappl| grid
+- :meth:`~pinefarm.external.interface.External.results`: provide runner
+  results on chosen PDF, for comparison with |pineappl| ``convolute`` ones
+- :meth:`~pinefarm.external.interface.External.annotate_versions`: collect
+  versions of all the program used to compute the results (for reproducibility)
+- :meth:`~pinefarm.external.interface.External.postprocess`: apply any
+  further step specified in :doc:`postprocess file <../pinecards/postrun>`, save
+  :doc:`metadata <../pinecards/metadata>`, and compress the final grid