LFP_analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LFP Analysis Pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook runs through the process of stimulus evoked LFP analysis step by step to the end figures. Please follow the upcoming steps to successfully analyze your data. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1) Copy your data from HDD to SSD \n",
    "\n",
    "First things first! Before any data analysis, we need to <b>copy the data from the hard drive where the raw data is stored to the 1 TB SSD on this computer that is designated as the workspace</b>. This is asked for two reasons: \n",
    "\n",
    "<ol type=\"1\">\n",
    "<li>There is a remarkable difference between the speed of reading data from the HDD vs. SSD</li>\n",
    "<li>There are some by-products of the data analysis that should be deleted at the end of the analysis. If these files are stored in the HDD, Dropbox tries to synchonize all those files as well. And also some times these files are forgotten to be deleted and end up taking precious space on the HDD where more novel, raw data could have been stored.</li>\n",
    "</ol>\n",
    "\n",
    "<b> IF </b> you have completed this step, please continue with the following steps."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2) Generate parameter dictionary for all recording sessions\n",
    "\n",
    "Next, we will need to generate pickle files named <i> paramsDict.p </i> for each recording session in your experiment. These pickle files contain crucial parameters related to data acquisition and your preferences on the details of how the data should be analyzed. For this procedure, please <b> run the block below </b> to launch the Jupyter notebook (Python 3) for generating the parameter dictionary. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "source activate klusta #Activating the klustakwik virtual environment\n",
    "jupyter notebook './Generate_dict_for_experiment.ipynb'\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  2.5) Restart kernel\n",
    "\n",
    "Since no bash or python command to this date exists for properly closing a jupyter notebook, from the <i> Kernel </i> tab above, select <i> Restart & Clear Output </i> to restart the kernel. That will not be an issue since the <i> paramsDict.p </i> files are already generated and saved."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3) Run the main analysis function on the data \n",
    "\n",
    "We are ready to perform the analysis on the data. Please first <b> specify the path (with / at the end) </b> to the folder that contains the folders for all recprding sessions (i.e. the folder right above the folders of recording sessions in hierarchy) and then <b> run the following line of code </b> to run the script <i> analyze_all_recording_sessions.py </i> which will run the <i> main </i> function in the <i> main_tetrode.py </i>. You can check the scripts for the details of the steps running on the background. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Currently analyzing:/Users/aagamshah/Data/2017_12_05_FUSs1_EphysM1_E-FUS_NBBB23/stim_0_3Hz_171205_162155\n",
      "start reading out and analyzing trodes\n",
      "  0%|                                                     | 0/2 [00:00<?, ?it/s]\n",
      "Reading Intan Technologies RHD2000 Data File, Version 1.5\n",
      "\n",
      "Found 32 amplifier channels.\n",
      "Found 3 auxiliary input channels.\n",
      "Found 1 supply voltage channel.\n",
      "Found 8 board ADC channels.\n",
      "Found 16 board digital input channels.\n",
      "Found 0 board digital output channels.\n",
      "Found 0 temperature sensors channels.\n",
      "\n",
      "Header file contains no data.  Amplifiers were sampled at 30.00 kS/s.\n",
      "Done!  Elapsed time: 0.0 seconds\n",
      "#### Low-pass and notch filtering the data ####\n",
      "\n",
      "  0%|                                                   | 0/324 [00:00<?, ?it/s]\u001b[A\n",
      " 20%|████████▏                                | 65/324 [00:00<00:00, 636.73it/s]\u001b[A\n",
      " 28%|███████████▍                             | 90/324 [00:00<00:00, 419.53it/s]\u001b[A\n",
      " 36%|██████████████▌                         | 118/324 [00:00<00:00, 372.25it/s]\u001b[A\n",
      " 50%|████████████████████                    | 163/324 [00:00<00:00, 390.56it/s]\u001b[A\n",
      " 65%|██████████████████████████▏             | 212/324 [00:00<00:00, 409.58it/s]\u001b[A\n",
      " 81%|████████████████████████████████▎       | 262/324 [00:00<00:00, 424.20it/s]\u001b[A\n",
      " 96%|██████████████████████████████████████▎ | 310/324 [00:00<00:00, 431.22it/s]\u001b[A\n",
      " 50%|██████████████████████                      | 1/2 [02:08<02:08, 128.45s/it]\u001b[A\n",
      "Reading Intan Technologies RHD2000 Data File, Version 1.5\n",
      "\n",
      "Found 32 amplifier channels.\n",
      "Found 3 auxiliary input channels.\n",
      "Found 1 supply voltage channel.\n",
      "Found 8 board ADC channels.\n",
      "Found 16 board digital input channels.\n",
      "Found 0 board digital output channels.\n",
      "Found 0 temperature sensors channels.\n",
      "\n",
      "Header file contains no data.  Amplifiers were sampled at 30.00 kS/s.\n",
      "Done!  Elapsed time: 0.0 seconds\n",
      "#### Low-pass and notch filtering the data ####\n",
      "\n",
      "  0%|                                                   | 0/324 [00:00<?, ?it/s]\u001b[A\n",
      " 41%|███████████████▉                       | 132/324 [00:00<00:00, 1318.67it/s]\u001b[A\n",
      " 84%|████████████████████████████████▋      | 272/324 [00:00<00:00, 1354.76it/s]\u001b[A\n",
      "100%|████████████████████████████████████████████| 2/2 [04:25<00:00, 132.51s/it]\u001b[A\n"
     ]
    }
   ],
   "source": [
    "#Specify the path here!\n",
    "PATHEXP=\"/Users/aagamshah/Data/2017_12_05_FUSs1_EphysM1_E-FUS_NBBB23/\" \n",
    "echo $PATHEXP|python analyze_all_recording_sessions.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4) Generate the figures and extract results\n",
    "\n",
    "As we have extracted the evoked LFP waveforms (and optionally ran the klusta, in case you enabled the spike sorting for this analysis), now we are ready to generate figures and  extract results for this experiment. These steps will be performed in the <i> evoked_lfp_analysis.py </i> script. Please <b> run the following line </b> to perform this step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "echo $PATHEXP | python evoked_lfp_analysis.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " # 5) Window LFP analysis (optional)\n",
    " \n",
    " If there is any recording session that you would like to analyze as broken down into time windows, please run the following block to open the ipython notebook for analyzing evoked LFP data in time windows. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "jupyter notebook './Window_LFP_analysis.ipynb'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Done! \n",
    "\n",
    "You can check the <i> Analyzed </i> folder to see the plots and the excel sheets containing the evoked LFP data and <b> move the files and folders </b> that you deem necessary into the <i> Analyzed </i> folder inside the <i> Electrophysiology </i> Dropbox folder. Please do not forget to rename the folder with the date and the name of the experiment when adding to the <i> Analyzed </i> folder. At the end, please <b> delete the data and the intermediate files from the SSD. </b> \n",
    "\n",
    "Notebook written by Baran Yasar in 04/2017. Please contact him in person or via e-mail at yasar@biomed.ee.ethz.ch in case of any questions."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}