Tutorials/Template_For_Notebooks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "409ef523",
   "metadata": {},
   "source": [
    "# Template for COSIMA Recipes\n",
    "\n",
    "This notebook shows the essential elements of a COSIMA recipe.\n",
    "\n",
    "**Requirements:** All notebooks need to run on the `conda/analysis3` kernel. This is the most recent stable release of the `conda` environment curated by the CLEX CMS team."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70f62e69",
   "metadata": {},
   "source": [
    "First, we load any required modules. The following are pretty standard:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "f677c18a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-08T06:50:24.329238Z",
     "iopub.status.busy": "2024-07-08T06:50:24.328889Z",
     "iopub.status.idle": "2024-07-08T06:50:47.006959Z",
     "shell.execute_reply": "2024-07-08T06:50:47.005737Z",
     "shell.execute_reply.started": "2024-07-08T06:50:24.329216Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import intake\n",
    "import numpy as np\n",
    "\n",
    "import xarray as xr\n",
    "import cf_xarray as cfxr\n",
    "\n",
    "from dask.distributed import Client"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63a9c0e0",
   "metadata": {},
   "source": [
    "These modules are more optional:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "fa471615",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-08T06:50:47.009122Z",
     "iopub.status.busy": "2024-07-08T06:50:47.008455Z",
     "iopub.status.idle": "2024-07-08T06:50:49.362108Z",
     "shell.execute_reply": "2024-07-08T06:50:49.361025Z",
     "shell.execute_reply.started": "2024-07-08T06:50:47.009084Z"
    }
   },
   "outputs": [],
   "source": [
    "import cmocean as cm                              # Nice colormaps\n",
    "from collections import OrderedDict               # We often use this to organise our experiments\n",
    "import cftime                                     # In case you need to work with time axes\n",
    "import glob                                       # If you need to search file systems\n",
    "import cartopy.crs as ccrs                        # For making maps\n",
    "import cartopy.feature as cft                     # For adding features to maps"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8501b098",
   "metadata": {},
   "source": [
    "It's often a good idea to start a cluster with multiple cores for you to work with. This is the easiest way to do that:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "72a30057",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-08T06:50:49.364772Z",
     "iopub.status.busy": "2024-07-08T06:50:49.363810Z",
     "iopub.status.idle": "2024-07-08T06:50:52.066296Z",
     "shell.execute_reply": "2024-07-08T06:50:52.065177Z",
     "shell.execute_reply.started": "2024-07-08T06:50:49.364714Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "    <div style=\"width: 24px; height: 24px; background-color: #e1e1e1; border: 3px solid #9D9D9D; border-radius: 5px; position: absolute;\"> </div>\n",
       "    <div style=\"margin-left: 48px;\">\n",
       "        <h3 style=\"margin-bottom: 0px;\">Client</h3>\n",
       "        <p style=\"color: #9D9D9D; margin-bottom: 0px;\">Client-690a1579-3cf6-11ef-86a0-0000018cfe80</p>\n",
       "        <table style=\"width: 100%; text-align: left;\">\n",
       "\n",
       "        <tr>\n",
       "        \n",
       "            <td style=\"text-align: left;\"><strong>Connection method:</strong> Cluster object</td>\n",
       "            <td style=\"text-align: left;\"><strong>Cluster type:</strong> distributed.LocalCluster</td>\n",
       "        \n",
       "        </tr>\n",
       "\n",
       "        \n",
       "            <tr>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Dashboard: </strong> <a href=\"/proxy/8787/status\" target=\"_blank\">/proxy/8787/status</a>\n",
       "                </td>\n",
       "                <td style=\"text-align: left;\"></td>\n",
       "            </tr>\n",
       "        \n",
       "\n",
       "        </table>\n",
       "\n",
       "        \n",
       "            <button style=\"margin-bottom: 12px;\" data-commandlinker-command=\"dask:populate-and-launch-layout\" data-commandlinker-args='{\"url\": \"/proxy/8787/status\" }'>\n",
       "                Launch dashboard in JupyterLab\n",
       "            </button>\n",
       "        \n",
       "\n",
       "        \n",
       "            <details>\n",
       "            <summary style=\"margin-bottom: 20px;\"><h3 style=\"display: inline;\">Cluster Info</h3></summary>\n",
       "            <div class=\"jp-RenderedHTMLCommon jp-RenderedHTML jp-mod-trusted jp-OutputArea-output\">\n",
       "    <div style=\"width: 24px; height: 24px; background-color: #e1e1e1; border: 3px solid #9D9D9D; border-radius: 5px; position: absolute;\">\n",
       "    </div>\n",
       "    <div style=\"margin-left: 48px;\">\n",
       "        <h3 style=\"margin-bottom: 0px; margin-top: 0px;\">LocalCluster</h3>\n",
       "        <p style=\"color: #9D9D9D; margin-bottom: 0px;\">d8c19fc4</p>\n",
       "        <table style=\"width: 100%; text-align: left;\">\n",
       "            <tr>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Dashboard:</strong> <a href=\"/proxy/8787/status\" target=\"_blank\">/proxy/8787/status</a>\n",
       "                </td>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Workers:</strong> 4\n",
       "                </td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Total threads:</strong> 4\n",
       "                </td>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Total memory:</strong> 15.00 GiB\n",
       "                </td>\n",
       "            </tr>\n",
       "            \n",
       "            <tr>\n",
       "    <td style=\"text-align: left;\"><strong>Status:</strong> running</td>\n",
       "    <td style=\"text-align: left;\"><strong>Using processes:</strong> True</td>\n",
       "</tr>\n",
       "\n",
       "            \n",
       "        </table>\n",
       "\n",
       "        <details>\n",
       "            <summary style=\"margin-bottom: 20px;\">\n",
       "                <h3 style=\"display: inline;\">Scheduler Info</h3>\n",
       "            </summary>\n",
       "\n",
       "            <div style=\"\">\n",
       "    <div>\n",
       "        <div style=\"width: 24px; height: 24px; background-color: #FFF7E5; border: 3px solid #FF6132; border-radius: 5px; position: absolute;\"> </div>\n",
       "        <div style=\"margin-left: 48px;\">\n",
       "            <h3 style=\"margin-bottom: 0px;\">Scheduler</h3>\n",
       "            <p style=\"color: #9D9D9D; margin-bottom: 0px;\">Scheduler-c3d0e4b3-1689-451e-961f-dd9e182dafdb</p>\n",
       "            <table style=\"width: 100%; text-align: left;\">\n",
       "                <tr>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Comm:</strong> tcp://127.0.0.1:33631\n",
       "                    </td>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Workers:</strong> 4\n",
       "                    </td>\n",
       "                </tr>\n",
       "                <tr>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Dashboard:</strong> <a href=\"/proxy/8787/status\" target=\"_blank\">/proxy/8787/status</a>\n",
       "                    </td>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Total threads:</strong> 4\n",
       "                    </td>\n",
       "                </tr>\n",
       "                <tr>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Started:</strong> Just now\n",
       "                    </td>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Total memory:</strong> 15.00 GiB\n",
       "                    </td>\n",
       "                </tr>\n",
       "            </table>\n",
       "        </div>\n",
       "    </div>\n",
       "\n",
       "    <details style=\"margin-left: 48px;\">\n",
       "        <summary style=\"margin-bottom: 20px;\">\n",
       "            <h3 style=\"display: inline;\">Workers</h3>\n",
       "        </summary>\n",
       "\n",
       "        \n",
       "        <div style=\"margin-bottom: 20px;\">\n",
       "            <div style=\"width: 24px; height: 24px; background-color: #DBF5FF; border: 3px solid #4CC9FF; border-radius: 5px; position: absolute;\"> </div>\n",
       "            <div style=\"margin-left: 48px;\">\n",
       "            <details>\n",
       "                <summary>\n",
       "                    <h4 style=\"margin-bottom: 0px; display: inline;\">Worker: 0</h4>\n",
       "                </summary>\n",
       "                <table style=\"width: 100%; text-align: left;\">\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Comm: </strong> tcp://127.0.0.1:40159\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Total threads: </strong> 1\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Dashboard: </strong> <a href=\"/proxy/34521/status\" target=\"_blank\">/proxy/34521/status</a>\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Memory: </strong> 3.75 GiB\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Nanny: </strong> tcp://127.0.0.1:46699\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\"></td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td colspan=\"2\" style=\"text-align: left;\">\n",
       "                            <strong>Local directory: </strong> /jobfs/119983945.gadi-pbs/dask-scratch-space/worker-t45b0mcf\n",
       "                        </td>\n",
       "                    </tr>\n",
       "\n",
       "                    \n",
       "\n",
       "                    \n",
       "\n",
       "                </table>\n",
       "            </details>\n",
       "            </div>\n",
       "        </div>\n",
       "        \n",
       "        <div style=\"margin-bottom: 20px;\">\n",
       "            <div style=\"width: 24px; height: 24px; background-color: #DBF5FF; border: 3px solid #4CC9FF; border-radius: 5px; position: absolute;\"> </div>\n",
       "            <div style=\"margin-left: 48px;\">\n",
       "            <details>\n",
       "                <summary>\n",
       "                    <h4 style=\"margin-bottom: 0px; display: inline;\">Worker: 1</h4>\n",
       "                </summary>\n",
       "                <table style=\"width: 100%; text-align: left;\">\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Comm: </strong> tcp://127.0.0.1:34449\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Total threads: </strong> 1\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Dashboard: </strong> <a href=\"/proxy/34065/status\" target=\"_blank\">/proxy/34065/status</a>\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Memory: </strong> 3.75 GiB\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Nanny: </strong> tcp://127.0.0.1:46145\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\"></td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td colspan=\"2\" style=\"text-align: left;\">\n",
       "                            <strong>Local directory: </strong> /jobfs/119983945.gadi-pbs/dask-scratch-space/worker-4x3s6gg2\n",
       "                        </td>\n",
       "                    </tr>\n",
       "\n",
       "                    \n",
       "\n",
       "                    \n",
       "\n",
       "                </table>\n",
       "            </details>\n",
       "            </div>\n",
       "        </div>\n",
       "        \n",
       "        <div style=\"margin-bottom: 20px;\">\n",
       "            <div style=\"width: 24px; height: 24px; background-color: #DBF5FF; border: 3px solid #4CC9FF; border-radius: 5px; position: absolute;\"> </div>\n",
       "            <div style=\"margin-left: 48px;\">\n",
       "            <details>\n",
       "                <summary>\n",
       "                    <h4 style=\"margin-bottom: 0px; display: inline;\">Worker: 2</h4>\n",
       "                </summary>\n",
       "                <table style=\"width: 100%; text-align: left;\">\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Comm: </strong> tcp://127.0.0.1:43443\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Total threads: </strong> 1\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Dashboard: </strong> <a href=\"/proxy/42517/status\" target=\"_blank\">/proxy/42517/status</a>\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Memory: </strong> 3.75 GiB\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Nanny: </strong> tcp://127.0.0.1:43835\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\"></td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td colspan=\"2\" style=\"text-align: left;\">\n",
       "                            <strong>Local directory: </strong> /jobfs/119983945.gadi-pbs/dask-scratch-space/worker-11wd9o42\n",
       "                        </td>\n",
       "                    </tr>\n",
       "\n",
       "                    \n",
       "\n",
       "                    \n",
       "\n",
       "                </table>\n",
       "            </details>\n",
       "            </div>\n",
       "        </div>\n",
       "        \n",
       "        <div style=\"margin-bottom: 20px;\">\n",
       "            <div style=\"width: 24px; height: 24px; background-color: #DBF5FF; border: 3px solid #4CC9FF; border-radius: 5px; position: absolute;\"> </div>\n",
       "            <div style=\"margin-left: 48px;\">\n",
       "            <details>\n",
       "                <summary>\n",
       "                    <h4 style=\"margin-bottom: 0px; display: inline;\">Worker: 3</h4>\n",
       "                </summary>\n",
       "                <table style=\"width: 100%; text-align: left;\">\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Comm: </strong> tcp://127.0.0.1:44797\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Total threads: </strong> 1\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Dashboard: </strong> <a href=\"/proxy/44577/status\" target=\"_blank\">/proxy/44577/status</a>\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Memory: </strong> 3.75 GiB\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Nanny: </strong> tcp://127.0.0.1:46405\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\"></td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td colspan=\"2\" style=\"text-align: left;\">\n",
       "                            <strong>Local directory: </strong> /jobfs/119983945.gadi-pbs/dask-scratch-space/worker-iyjlykyq\n",
       "                        </td>\n",
       "                    </tr>\n",
       "\n",
       "                    \n",
       "\n",
       "                    \n",
       "\n",
       "                </table>\n",
       "            </details>\n",
       "            </div>\n",
       "        </div>\n",
       "        \n",
       "\n",
       "    </details>\n",
       "</div>\n",
       "\n",
       "        </details>\n",
       "    </div>\n",
       "</div>\n",
       "            </details>\n",
       "        \n",
       "\n",
       "    </div>\n",
       "</div>"
      ],
      "text/plain": [
       "<Client: 'tcp://127.0.0.1:33631' processes=4 threads=4, memory=15.00 GiB>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client = Client()\n",
    "client"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdb7b4be",
   "metadata": {},
   "source": [
    "You will need to nominate a catalog from which to load your data. Unless there is a pressing need, it is best to use the default catalog:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "036a21ac",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-08T06:50:52.069842Z",
     "iopub.status.busy": "2024-07-08T06:50:52.068610Z",
     "iopub.status.idle": "2024-07-08T06:50:53.578404Z",
     "shell.execute_reply": "2024-07-08T06:50:53.577171Z",
     "shell.execute_reply.started": "2024-07-08T06:50:52.069788Z"
    }
   },
   "outputs": [],
   "source": [
    "catalog = intake.cat.access_nri"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6990b795",
   "metadata": {},
   "source": [
    "**Now you are ready to begin your custom code.** This will invariably require:\n",
    "* finding data using ```catalog.search()```\n",
    "* processing or manipulating the data\n",
    "* making one or more plots.\n",
    "We recommend that, at a minimum, these 3 tasks are done in 3 separate cells, but many examples will use multiple cells in the processing step. \n",
    "\n",
    "At all stages you should **try your best to document your code**. As a rule of thumb, if an undergraduate student can follow what you have done, then your documentation is about right ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "783a632e-ed26-4fc2-822d-1664c1c7d545",
   "metadata": {},
   "source": [
    "Its best to also close your cluster at the end of the notebook:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "594ae08a-e784-431d-b709-862bb49a582c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-08T06:50:53.580708Z",
     "iopub.status.busy": "2024-07-08T06:50:53.580176Z",
     "iopub.status.idle": "2024-07-08T06:50:53.822675Z",
     "shell.execute_reply": "2024-07-08T06:50:53.821930Z",
     "shell.execute_reply.started": "2024-07-08T06:50:53.580656Z"
    }
   },
   "outputs": [],
   "source": [
    "client.close()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:analysis3]",
   "language": "python",
   "name": "conda-env-analysis3-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}