Merge branch 'latest' into load_signatures_40

.github/workflows/dev_envs.yml

@@ -0,0 +1,64 @@
+name: "Dev env instructions"
+on:
+  pull_request:
+    branches: [latest]
+  push:
+    branches: [latest]
+jobs:
+  nix:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2.3.4
+      with:
+        fetch-depth: 0
+
+    - name: Cache nix store
+      id: cache-nix
+      uses: actions/cache@v2
+      with:
+        path: /nix/store
+        key: nix-${{ hashFiles('shell.nix') }}-${{ hashFiles('nix/**') }}
+
+    - uses: cachix/install-nix-action@v12
+      with:
+        nix_path: nixpkgs=channel:nixos-20.09
+
+    - run: nix-shell --command "tox -e py39"
+
+  mamba:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v2.3.4
+      with:
+        fetch-depth: 0
+
+    - name: cache conda
+      uses: actions/cache@v1
+      env:
+        CACHE_NUMBER: 0
+      with:
+        path: ~/conda_pkgs_dir
+        key:
+          ${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{ hashFiles('environment.yml') }}
+
+    - name: setup conda
+      uses: conda-incubator/setup-miniconda@e23d871804685e8c52189e5bd45e9145019f10af
+      with:
+        auto-update-conda: true
+        python-version: 3.9
+        channels: conda-forge,bioconda
+        miniforge-variant: Mambaforge
+        miniforge-version: latest
+        use-mamba: true
+        mamba-version: "*"
+        activate-environment: sourmash_dev
+        auto-activate-base: false
+        use-only-tar-bz2: true
+
+    - name: install dependencies
+      shell: bash -l {0}
+      run: mamba install tox-conda rust git compilers pandoc
+
+    - name: run tests for 3.9
+      shell: bash -l {0}
+      run: tox -e py39

doc/developer.md

@@ -6,43 +6,79 @@ You can get the latest development branch with:
 ```
 git clone https://github.com/dib-lab/sourmash.git
 ```
-sourmash runs under Python 3.7 and later.  The base
-requirements are screed and cffi, together with a Rust environment (for the
-extension code). We suggest using `rustup` to install the Rust environment:
+sourmash runs under Python 3.7 and later.
 
-    curl https://sh.rustup.rs -sSf | sh
+We recommend using `conda` or `Nix` for setting up an environment for developing
+new features, running tests and code quality checks.
+Here are some suggestions on how to set them up (note: you only need one =])
 
-We use [`tox`](https://tox.readthedocs.io) for managing dependencies and
-running tests and checks during development.
-To install it, do:
+### Using mamba (conda alternative)
+
+Follow the [installation instructions](https://github.com/conda-forge/miniforge#install) for
+installing `mambaforge` (a conda distribution that uses
+[`mamba`](https://github.com/TheSnakePit/mamba)
+and the [`conda-forge`](https://conda-forge.org/) channel by default).
+
+Once `mamba` is installed, run
+```
+mamba create -n sourmash_dev tox-conda rust git compilers pandoc
+```
+to create an environment called `sourmash_dev` containing the programs needed
+for development.
+
+To activate the new environment, run
+```
+conda activate sourmash_dev
+```
+and proceed to the ["Running tests and checks"](#running-tests-and-checks) section.
+
+### Using Nix
+
+Follow the [installation instructions](https://nixos.org/manual/nix/stable/#chap-installation)
+for setting up Nix in your system (Linux or macOS).
+
+Once Nix is installed, run
+```
+nix-shell
+```
+to start an environment ready for [running tests and checks](#running-tests-and-checks).
+
+### General instructions
+
+As long as you have `tox` and a Rust compiler available,
+you can skip `mamba` or `Nix`.
+
+For Rust, we suggest using `rustup` to install the Rust environment:
+```
+curl https://sh.rustup.rs -sSf | sh
+```
+And for `tox` you can run
 ```
 python -m pip install tox
 ```
-and use `tox -l` to list available tasks.
 
 We suggest working on sourmash in a virtualenv; e.g. from within the
-sourmash clone directory, you can do:
+cloned repository (and after installing `tox` and Rust), you can do:
 ```
 tox -e dev
 . .tox/dev/bin/activate
 ```
 
-You can run tests by invoking `make test` in the sourmash directory;
-`tox -e py39` will run the Python tests with Python 3.9,
-and `cargo test` will run the Rust tests.
-
-You can also explicitly install all the dependencies for sourmash by running
+Finally, ou can also explicitly install all the Python dependencies for sourmash by running
 ```
 pip install -r requirements.txt
 ```
+(but they are already installed in the virtualenv created with `tox -e dev`).
 
-### If you're having trouble installing or using the development environment
+## Running tests and checks
 
-If you are getting an error that contains `ImportError: cannot import name 'to_bytes' from 'sourmash.minhash'`, then it's likely you need to update Rust and clean up your environment. Some installation issues can be solved by simply removing the intermediate build files with: 
+We use [`tox`](https://tox.readthedocs.io) for managing dependencies and
+running tests and checks during development.
+`tox -l` lists available tasks.
 
-```
-make clean
-```
+You can run tests by invoking `make test` in the sourmash directory;
+`tox -e py39` will run the Python tests with Python 3.9,
+and `cargo test` will run the Rust tests.
 
 ## Adding new changes
 
@@ -99,7 +135,7 @@ A short description of the high-level files and dirs in the sourmash repo:
 ├── Makefile            | Entry point for most development tasks
 ├── MANIFEST.in         | Describes what files to add to the Python package
 ├── matplotlibrc        | Configuration for matplotlib
-├── nix.shell           | Nix configuration for creating a dev environment
+├── shell.nix           | Nix configuration for creating a dev environment
 ├── paper.bib           | References in the JOSS paper
 ├── paper.md            | JOSS paper content
 ├── pyproject.toml      | Python project definitions (build system and tooling)
@@ -218,6 +254,18 @@ For the Rust core library we use `rMAJOR.MINOR.PATCH`
 The Rust version is not automated,
 and must be bumped in `src/core/Cargo.toml`.
 
+## Common errors and solutions
+
+### Cannot import name `to_bytes` from `sourmash.minhash`
+
+If you are getting an error that contains `ImportError: cannot import name 'to_bytes' from 'sourmash.minhash'`,
+then it's likely you need to update Rust and clean up your environment.
+Some installation issues can be solved by simply removing the intermediate build files with:
+
+```
+make clean
+```
+
 ## Contents
 
 ```{toctree}

nix/sources.json

@@ -0,0 +1,38 @@
+{
+    "niv": {
+        "branch": "master",
+        "description": "Easy dependency management for Nix projects",
+        "homepage": "https://github.com/nmattia/niv",
+        "owner": "nmattia",
+        "repo": "niv",
+        "rev": "3cd7914b2c4cff48927e11c216dadfab7d903fe5",
+        "sha256": "1agq4nvbhrylf2s77kb4xhh9k7xcwdwggq764k4jgsbs70py8cw3",
+        "type": "tarball",
+        "url": "https://github.com/nmattia/niv/archive/3cd7914b2c4cff48927e11c216dadfab7d903fe5.tar.gz",
+        "url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
+    },
+    "nixpkgs": {
+        "branch": "nixpkgs-unstable",
+        "description": "Nix Packages collection",
+        "homepage": "",
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "d1f97a5eb5115289071d8449f26e7b92ce6b6709",
+        "sha256": "098rb747w4p5lxz74bj738bpjd4xjn0ahssjp9n8237dmmx3wg5p",
+        "type": "tarball",
+        "url": "https://github.com/NixOS/nixpkgs/archive/d1f97a5eb5115289071d8449f26e7b92ce6b6709.tar.gz",
+        "url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
+    },
+    "rust-overlay": {
+        "branch": "master",
+        "description": null,
+        "homepage": null,
+        "owner": "oxalica",
+        "repo": "rust-overlay",
+        "rev": "0bb9ef6d8b34e5579d7384201f3106a49ce3deca",
+        "sha256": "09cva1r53x1ihyzbssdscx4n5913pra8f006q095ww9wvfvz8bxf",
+        "type": "tarball",
+        "url": "https://github.com/oxalica/rust-overlay/archive/0bb9ef6d8b34e5579d7384201f3106a49ce3deca.tar.gz",
+        "url_template": "https://github.com/<owner>/<repo>/archive/<rev>.tar.gz"
+    }
+}