This version of B-SOiD has been modified to enable running through the CLI. It has only been tested using CSV files from DeepLabCut with corresponding .mp4 videos. We have tried to enable the other file types used in the original code, but cannot guarantee they will work smoothly. The usage section has been modified to reflect the CLI usage. We have not included functionality for launching bsoid_analysis.py (the final step in the original pipeline).
DeepLabCut 1,2,3, SLEAP 4, and OpenPose 5 have revolutionized the way behavioral scientists analyze data. These algorithm utilizes recent advances in computer vision and deep learning to automatically estimate 3D-poses. Interpreting the positions of an animal can be useful in studying behavior; however, it does not encompass the whole dynamic range of naturalistic behaviors.
B-SOiD identifies behaviors using a unique pipeline where unsupervised learning meets supervised classification. The unsupervised behavioral segmentation relies on non-linear dimensionality reduction 6,7,9,10, whereas the supervised classification is standard scikit-learn 8.
Behavioral segmentation of open field in DeepLabCut, or B-SOiD ("B-side"), as the name suggested, was first designed as a pipeline using pose estimation file from DeepLabCut as input. Now, it has extended to handle DeepLabCut (.h5, .csv), SLEAP (.h5), and OpenPose (.json) files.
This repository uses conda to manage the development software environment.
You can find operating system-specific instructions for installing miniconda here. After installing conda and mamba, proceed with the next steps.
Git clone the web URL (example below) or download ZIP.
Change your current working directory to the location where you want the cloned directory to be made.
git clone https://github.com/Arcadia-Science/B-SOID.gitStep 1: Setup, open an anaconda/python3 instance and install dependencies with the requirements file
cd /path/to/B-SOID/
For MacOS users (note: you need a Mac with an Intel chip or use the Rosetta emulator on an Apple silicon based Macs):
conda env create -n bsoid_v2 -f requirements.yaml (macOS)or for Windows users:
conda env create -n bsoid_v2 -f requirements_win.yaml (windows)or for Linux users (this was tested with Ubuntu version 22.2):
Linux ffmpeg distribution on Conda doesn't work (it has missing libraries). Because of this, you need to install ffmpeg separately from the Conda environment setup. First, let's do that:
sudo apt update && sudo apt upgrade
sudo apt install ffmpeg # At this time, the most recent ffmpeg version is 4.2.2Then let's create the Conda environment. Sadly Conda/Mamba is not able to resolve the dependencies when the channel_priority setting is set to strict. So, we need to change that:
# Adjust the channel_priority setting
conda config --set channel_priority flexible
# Create the conda environment
conda env create -n bsoid_v2 -f requirements_linux.yaml
# Optionally update the channel_priority setting to be strict
conda config --set channel_priority strictOnce your setup is complete, activate the Conda environment:
conda activate bsoid_v2You should now see (bsoid_v2) $yourusername@yourmachine ~ %
Note: If you are running on an AWS instance and would like to see outputs from the web application, you will need to add an inbound rule to your security group with Type: “Custom TCP Rule”, Port Range:8501, and Source: MyIP. Seeing the web application can be useful for debugging since errors sometimes only appear there. Port 8501 is the custom port used by Streamlit. However, users should be careful to make sure that this is the right port when they run the app; for example, if you incorrectly shutdown so the process is running the background and you run the streamlit app again, it will select the next port.
WORKING_DIR_BSOID: Path to where you want outputs to go.
PREFIX_BSOID: Prefix that you want appended to output files.
SOFTWARE_BSOID: Pose estimation software; must be 'DeepLabCut','SLEAP', or 'OpenPose'.
FTYPE_BSOID: File type; DeepLabCut: 'h5' or 'csv', SLEAP: 'h5', and OpenPose: 'json'.
ROOT_PATH_BSOID: Path to the working directory containing sub-directories that have input .csv,.h5, or .json files.
FRAMERATE_BSOID: Framerate for pose estimate files; in frames per second.
VALUE_BSOID: Number of sub-directories that have input .csv,.h5, or .json files.
DATA_DIR_BSOID: Path of sub-directories that have input .csv,.h5, or .json files; provided as a list and relative to working_dir, e.g. '/1_1,/1_2,/2_1,/2_2'.
MIN_CLUSTER_BSOID: default='0.5',Minimum cluster size, based on minimum temporal bout and will represent a %. Impacts number of clusters.
MAX_CLUSTER_BSOID: default='1', Maximum cluster size, will represent a % and impacts number of clusters.
AUTOSAVE_BSOID: Whether or not you want to autosave clustering as you go. Should be 'Yes' or 'No'. Default is 'Yes'.
POSE_LIST_BSOID: List of poses to include in analysis, for example, 'R_rear,L_rear'.
MIN_TIME_BSOID: Minimum time for bout in ms. Default is 200.
NUMBER_EXAMPLES_BSOID: Number of non-repeated examples for video snippets. Default is 5. Decreasing this number will speed up runtime because fewer example gifs will be generated per cluster.
PLAYBACK_SPEED_BSOID: Playback speed for video snippets. Default is 0.75X.
FRACTION_BSOID: Training input fraction (do not change this value if you wish to generate the side-by-side video seen on B-SOiD GitHub page). Default is 1, minimum is 0.1, maximum is 1.
SOFTWARE_BSOID='DeepLabCut' FTYPE_BSOID='csv' ROOT_PATH_BSOID='/Users/Desktop/training/' FRAMERATE_BSOID=120 WORKING_DIR_BSOID='/Users/Desktop/training/output' PREFIX_BSOID='controltry' NUMBER_EXAMPLES_BSOID=1 VALUE_BSOID=4.0 DATA_DIR_BSOID='/1_1,/1_2,/2_1,/2_2' AUTOSAVE_BSOID='Yes' POSE_LIST_BSOID='R_rear,L_rear' python run_streamlit_cli.py
Your environmental variables should be set to match the directories that contain the trained model. Only WORKING_DIR_BSOID, PREFIX_BSOID, FTYPE_BSOID, ROOT_PATH_BSOID, FRAMERATE_BSOID, and DATA_DIR_BSOID need to be defined for prediction. Outputs from prediction will be within those directories in a folder named BSOID.
FTYPE_BSOID='csv' ROOT_PATH_BSOID='/Users/Desktop/training/' FRAMERATE_BSOID=120 WORKING_DIR_BSOID='/Users/Desktop/training/output' PREFIX_BSOID='controltry' DATA_DIR_BSOID='/1_1,/1_2,/2_1,/2_2' python run_streamlit_cli_predict.py
We have provided our 6 body part DeepLabCut model. We also included two example 5 minute clips (labeled_clip1, labeled_clip2) as proxy for how well we trained our model. The raw video (raw_clip1, raw_clip2) and the corresponding h5/pickle/csv files are included as well.
Pull requests are welcome. For recommended changes that you would like to see, open an issue. Join our slack group for more instantaneous feedback.
There are many exciting avenues to explore based on this work. Please do not hesitate to contact us for collaborations.
This software package provided without warranty of any kind and is licensed under the GNU General Public License v3.0. If you use our algorithm and/or model/data, please cite us! Preprint/peer-review will be announced in the following section.
September 2019: First B-SOiD preprint on bioRxiv
March 2020: Updated version of our preprint on bioRxiv