cytograph-shoji

Installation

The following instructions should work for Linux and Mac (unfortunately, we have no experience with Windows).

1. Install Anaconda

2. Install Shoji

3. Install cytograph-shoji:

git clone https://github.com/linnarsson-lab/cytograph-shoji.git
pip install -e cytograph-shoji

Getting started

First create a build folder (this is where you will configure the build and see the output):

$ mkdir /proj/cytograph/<yourname>/<buildname>  # Replace with your name and the name of the build
$ cd /proj/cytograph/<yourname>/<buildname>

Initialize the build folder. This will also create a corresponding workspace in Shoji.

$ cytograph init

You will see that your build folder now looks like this:

config.yaml       # Configuration and recipes
logs/             # Log files (when running on condor)
exported/         # Plots will appear here
punchcards/       # Punchards define individual analyses
  Example.yaml    # An example punchcard to get you started

You can look at the config.yaml and Example.yaml files to understand the default configuration. The files are extensively commented to help you understand what each option does.

To process the Example.yaml punchcard, you must first run the qc subcommand. This will run the qc recipe (in config.yaml) on each sample mentioned as a source in the Example.yaml punchcard:

$ cytograph qc --force Example   # Note: you can also name a specific sample
Cytograph v3.0.0 by Linnarsson Lab 🌸 (http://linnarssonlab.org)

2022-02-17 17:30:41,386 - INFO - Workspace is 'test'
2022-02-17 17:30:41,398 - INFO - Build folder is '/Users/stelin/cytograph/test'
2022-02-17 17:30:41,445 - INFO - Processing 'TenX89_1'
...etc...

We used `--force' to force a rerun of the qc recipe. Once you have run qc on a sample you don't need to do it again unless the recipe changed.

Now take a look in the exported/qc folder; you will find there the qc plots for each sample.

Once all the samples have been qc'ed, you can process the punchcard:

$ cytograph process Example
Cytograph v3.0.0 by Linnarsson Lab 🌸 (http://linnarssonlab.org)

2022-02-17 17:49:21,254 - INFO - Workspace is 'test'
2022-02-17 17:49:21,266 - INFO - Build folder is '/Users/stelin/cytograph/test'
2022-02-17 17:49:21,346 - WARNING - Deleting existing Workspace 'builds.cytograph.test.Example'
2022-02-17 17:49:21,495 - INFO - Creating Workspace 'builds.cytograph.test.Example'
2022-02-17 17:49:21,630 - INFO - Processing 'Example'
2022-02-17 17:49:21,638 - INFO - Running recipe 'example' for 'Example'
2022-02-17 17:49:21,638 - INFO - InitializeWorkspace: {'from_workspace': 'samples.TenX89_1', 'tensors': ['Gene', 'Accession', 'Chromosome', 'Start', 'End']}
2022-02-17 17:49:21,695 - INFO -  InitializeWorkspace: Collecting tensors from 'samples.TenX89_1'
2022-02-17 17:49:33,060 - INFO - InitializeWorkspace: Done in 11s.
2022-02-17 17:49:33,061 - INFO - 
...etc...

Notice that a new build is created in shoji under builds.cytograph.test.Example. That's where you will find the actual data generated by the run. Logs and plots will be saved in the build folder /Users/stelin/cytograph/test.

This completes your first cytograph-shoji punchcard analysis.

Other things

Rebuilding the docs

Make sure you have pdoc3 installed. In the repo, run:

pdoc --html cytograph --force

This will regenerate the API docs.

Troubleshooting

If, when importing cytograph in python, you get errors related to imports from 'harmony', solve by:

pip install harmony-pytorch

(further reading on https://pypi.org/project/harmony-pytorch/)

If OpenTSNE does not use more than one CPU on macOS, try installing llvm using Homebrew:

brew install llvm export LDFLAGS="-L/usr/local/opt/llvm/lib" export CPPFLAGS="-I/usr/local/opt/llvm/include" export PATH="/usr/local/opt/llvm/bin:$PATH"