big_data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "expired-force",
   "metadata": {},
   "source": [
    "#  Large data\n",
    "\n",
    "> questions:\n",
    "> - \"How do I work with multiple CMIP files that won't fit in memory?\"\n",
    ">\n",
    "> objectives:\n",
    "> - \"Import the dask library and start a client with parallel workers.\"\n",
    "> - \"Inspect netCDF chunking.\"\n",
    "> - \"Calculate and plot an ocean temperature climatology.\"\n",
    ">\n",
    "> keypoints:\n",
    "> - \"Libraries such as dask and xarray can make loading, processing and visualising netCDF data much easier.\"\n",
    "> - \"Dask can massively speed up processing through parallelism but care may be needed particularly with data chunking.\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "given-malta",
   "metadata": {},
   "source": [
    "> **Data used**\n",
    ">\n",
    ">If teaching this lesson in a classroom, a copy of the dataset should be on hand on external media such as a USB drive in case of wifi limitations. For remote teaching, please note the data being used is quite large, if network issues arise, the participant should instead use the smaller precipitation data files from the previous chapters."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "according-radio",
   "metadata": {},
   "source": [
    "So far we've been working with small, individual data files that can be comfortably read into memory on a modern laptop. What if we wanted to process a larger dataset that consists of many files and/or much larger file sizes? For instance, let's say we are studying the influence of sea surface temperature (SST) on precipitation. Now that we've plotted a global map showing the ACCESS-CM2 precipitation climatology, our next step is to create a similar global map showing the daily maximum SST over the 1850-2014 period.\n",
    "\n",
    "Rather than download all the ACCESS-CM2 daily SST files to our laptop, for the purposes of this lesson we are going to make use of the fact that the National Computational Infrastructure (NCI) in Canberra has made their archive of CMIP6 data remotely available via a \"THREDDS\" (or TDS) server. A THREDDS server provides access to OPeNDAP, which is a protocol to remotely access netCDF data over a network as though it were a local file on your computer.\n",
    "\n",
    "> TODO: On the instructor notes page maybe we should encourage instructors who aren't in Australia to use an alternative ESGF node (if the others nodes have THREDDS servers too)?\n",
    "\n",
    "In Python, we use the `siphon` library to query THREDDS catalogues to find available files.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "thermal-genre",
   "metadata": {},
   "outputs": [],
   "source": [
    "from siphon.catalog import TDSCatalog"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "unique-retail",
   "metadata": {},
   "outputs": [],
   "source": [
    "cat = TDSCatalog(\"http://dapds00.nci.org.au/thredds/catalog/fs38/publications/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/latest/catalog.xml\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "lucky-separation",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_18500101-18591231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_18600101-18691231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_18700101-18791231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_18800101-18891231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_18900101-18991231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19000101-19091231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19100101-19191231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19200101-19291231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19300101-19391231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19400101-19491231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19500101-19591231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19600101-19691231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19700101-19791231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19800101-19891231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19900101-19991231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_20000101-20091231.nc\n",
      "tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_20100101-20141231.nc\n"
     ]
    }
   ],
   "source": [
    "print(\"\\n\".join(cat.datasets.keys()))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "accepted-corpus",
   "metadata": {},
   "source": [
    "We can see that daily time scale SST data is spread across 17 data files. To access those files, we need to append the appropriate URL to the file names:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "ranking-windsor",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_18500101-18591231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_18600101-18691231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_18700101-18791231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_18800101-18891231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_18900101-18991231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19000101-19091231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19100101-19191231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19200101-19291231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19300101-19391231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19400101-19491231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19500101-19591231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19600101-19691231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19700101-19791231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19800101-19891231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_19900101-19991231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_20000101-20091231.nc', 'https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/tos_Oday_ACCESS-CM2_historical_r1i1p1f1_gn_20100101-20141231.nc']\n"
     ]
    }
   ],
   "source": [
    "file_list=list(cat.datasets.keys())\n",
    "DAProot='https://esgf.nci.org.au/thredds/dodsC/master/CMIP6/CMIP/CSIRO-ARCCSS/ACCESS-CM2/historical/r1i1p1f1/Oday/tos/gn/v20191108/'\n",
    "accesscm2_tos_file_list = [ DAProot+f for f in file_list ]\n",
    "print(accesscm2_tos_file_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "beginning-thirty",
   "metadata": {},
   "source": [
    "Now that we've got our list of files, we can use `xarray` to open a \"multifile\" dataset as though it were a single file.\n",
    "\n",
    "Recall that when we first open data in `xarray` it simply (\"lazily\") loads the metadata associated with the data and shows summary information about the contents of the dataset (i.e. it doesn't read the actual data into memory).\n",
    "**This may take a little time for a large multifile dataset!**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "editorial-husband",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<xarray.Dataset>\n",
      "Dimensions:             (bnds: 2, i: 360, j: 300, time: 60265, vertices: 4)\n",
      "Coordinates:\n",
      "  * time                (time) datetime64[ns] 1850-01-01T12:00:00 ... 2014-12...\n",
      "  * j                   (j) int32 0 1 2 3 4 5 6 ... 293 294 295 296 297 298 299\n",
      "  * i                   (i) int32 0 1 2 3 4 5 6 ... 353 354 355 356 357 358 359\n",
      "    latitude            (j, i) float64 dask.array<chunksize=(300, 360), meta=np.ndarray>\n",
      "    longitude           (j, i) float64 dask.array<chunksize=(300, 360), meta=np.ndarray>\n",
      "Dimensions without coordinates: bnds, vertices\n",
      "Data variables:\n",
      "    time_bnds           (time, bnds) datetime64[ns] dask.array<chunksize=(3652, 2), meta=np.ndarray>\n",
      "    vertices_latitude   (time, j, i, vertices) float64 dask.array<chunksize=(3652, 300, 360, 4), meta=np.ndarray>\n",
      "    vertices_longitude  (time, j, i, vertices) float64 dask.array<chunksize=(3652, 300, 360, 4), meta=np.ndarray>\n",
      "    tos                 (time, j, i) float32 dask.array<chunksize=(166, 300, 360), meta=np.ndarray>\n",
      "Attributes:\n",
      "    Conventions:                     CF-1.7 CMIP-6.2\n",
      "    activity_id:                     CMIP\n",
      "    branch_method:                   standard\n",
      "    branch_time_in_child:            0.0\n",
      "    branch_time_in_parent:           0.0\n",
      "    creation_date:                   2019-11-08T18:23:57Z\n",
      "    data_specs_version:              01.00.30\n",
      "    experiment:                      all-forcing simulation of the recent past\n",
      "    experiment_id:                   historical\n",
      "    external_variables:              areacello\n",
      "    forcing_index:                   1\n",
      "    frequency:                       day\n",
      "    further_info_url:                https://furtherinfo.es-doc.org/CMIP6.CSI...\n",
      "    grid:                            native atmosphere N96 grid (144x192 latx...\n",
      "    grid_label:                      gn\n",
      "    history:                         2019-11-08T18:23:57Z ; CMOR rewrote data...\n",
      "    initialization_index:            1\n",
      "    institution:                     CSIRO (Commonwealth Scientific and Indus...\n",
      "    institution_id:                  CSIRO-ARCCSS\n",
      "    mip_era:                         CMIP6\n",
      "    nominal_resolution:              250 km\n",
      "    notes:                           Exp: CM2-historical; Local ID: bj594; Va...\n",
      "    parent_activity_id:              CMIP\n",
      "    parent_experiment_id:            piControl\n",
      "    parent_mip_era:                  CMIP6\n",
      "    parent_source_id:                ACCESS-CM2\n",
      "    parent_time_units:               days since 0950-01-01\n",
      "    parent_variant_label:            r1i1p1f1\n",
      "    physics_index:                   1\n",
      "    product:                         model-output\n",
      "    realization_index:               1\n",
      "    realm:                           ocean\n",
      "    run_variant:                     forcing: GHG, Oz, SA, Sl, Vl, BC, OC, (G...\n",
      "    source:                          ACCESS-CM2 (2019): \\naerosol: UKCA-GLOMA...\n",
      "    source_id:                       ACCESS-CM2\n",
      "    source_type:                     AOGCM\n",
      "    sub_experiment:                  none\n",
      "    sub_experiment_id:               none\n",
      "    table_id:                        Oday\n",
      "    table_info:                      Creation Date:(30 April 2019) MD5:e14f55...\n",
      "    title:                           ACCESS-CM2 output prepared for CMIP6\n",
      "    variable_id:                     tos\n",
      "    variant_label:                   r1i1p1f1\n",
      "    version:                         v20191108\n",
      "    cmor_version:                    3.4.0\n",
      "    _NCProperties:                   version=2,netcdf=4.6.2,hdf5=1.10.5\n",
      "    tracking_id:                     hdl:21.14100/45e464ff-5f78-4b7b-8924-f3b...\n",
      "    license:                         CMIP6 model data produced by CSIRO is li...\n",
      "    DODS_EXTRA.Unlimited_Dimension:  time\n"
     ]
    }
   ],
   "source": [
    "import xarray as xr\n",
    "\n",
    "dset = xr.open_mfdataset(accesscm2_tos_file_list, combine='by_coords', chunks={'time':'100MB'})\n",
    "print(dset)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "deluxe-honey",
   "metadata": {},
   "source": [
    "We can see that our `dset` object is an `xarray.Dataset`, but notice now that each variable has type `dask.array`, meaning that `xarray` is aware of the netCDF \"chunks\" (how the data is packed in the files). We'll be able to parallelise our data processing across these chunks if we need/want to. To keep the chunks relatively small (to avoid THREDDS download limits), when calling `open_mfdataset` we set a limit on the size of the chunks with `chunks={'time':'100MB'}`.\n",
    "\n",
    "In this case, we are interested in the SST (`tos`) variable contained within that `xarray` Dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "explicit-bristol",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<xarray.DataArray 'tos' (time: 60265, j: 300, i: 360)>\n",
      "dask.array<concatenate, shape=(60265, 300, 360), dtype=float32, chunksize=(231, 300, 360), chunktype=numpy.ndarray>\n",
      "Coordinates:\n",
      "  * time       (time) datetime64[ns] 1850-01-01T12:00:00 ... 2014-12-31T12:00:00\n",
      "  * j          (j) int32 0 1 2 3 4 5 6 7 8 ... 292 293 294 295 296 297 298 299\n",
      "  * i          (i) int32 0 1 2 3 4 5 6 7 8 ... 352 353 354 355 356 357 358 359\n",
      "    latitude   (j, i) float64 dask.array<chunksize=(300, 360), meta=np.ndarray>\n",
      "    longitude  (j, i) float64 dask.array<chunksize=(300, 360), meta=np.ndarray>\n",
      "Attributes:\n",
      "    standard_name:  sea_surface_temperature\n",
      "    long_name:      Sea Surface Temperature\n",
      "    comment:        Temperature of upper boundary of the liquid ocean, includ...\n",
      "    units:          degC\n",
      "    cell_methods:   area: mean where sea time: mean\n",
      "    cell_measures:  area: areacello\n",
      "    history:        2019-11-08T18:23:54Z altered by CMOR: replaced missing va...\n",
      "    _ChunkSizes:    [  1 300 360]\n"
     ]
    }
   ],
   "source": [
    "print(dset['tos'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "italian-actor",
   "metadata": {},
   "source": [
    "Notice that we now have an attribute `_ChunkSizes` listed. This has shape `[1 300 360]`, while the `dask.array` itself has shape (60265, 300, 360), and chunksize (231, 300, 360). This means that the underlying data is structured to be most efficiently accessed for the whole lat/lon range at each time step, but dask will load up 231 of these \"slices\" at once, for a combined dataset size of 60265 timesteps.\n",
    "\n",
    "> **Changing chunks**\n",
    ">\n",
    "> If we decide to change the chunking to improve performance, note we\n",
    "> can control the size of dask chunks used, but they *must* align\n",
    "> with the netCDF file chunks or we will certainly make performance worse!\n",
    "{: .callout}\n",
    "\n",
    "Now that we understand the chunking information contained in the metadata, let's go ahead and calculate the dailiy maximum SST."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "ideal-multimedia",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<xarray.DataArray 'tos' (j: 300, i: 360)>\n",
      "dask.array<nanmax-aggregate, shape=(300, 360), dtype=float32, chunksize=(300, 360), chunktype=numpy.ndarray>\n",
      "Coordinates:\n",
      "  * j          (j) int32 0 1 2 3 4 5 6 7 8 ... 292 293 294 295 296 297 298 299\n",
      "  * i          (i) int32 0 1 2 3 4 5 6 7 8 ... 352 353 354 355 356 357 358 359\n",
      "    latitude   (j, i) float64 dask.array<chunksize=(300, 360), meta=np.ndarray>\n",
      "    longitude  (j, i) float64 dask.array<chunksize=(300, 360), meta=np.ndarray>\n",
      "Attributes:\n",
      "    standard_name:  sea_surface_temperature\n",
      "    long_name:      Sea Surface Temperature\n",
      "    comment:        Temperature of upper boundary of the liquid ocean, includ...\n",
      "    units:          degC\n",
      "    cell_methods:   area: mean where sea time: mean\n",
      "    cell_measures:  area: areacello\n",
      "    history:        2019-11-08T18:23:54Z altered by CMOR: replaced missing va...\n",
      "    _ChunkSizes:    [  1 300 360]\n"
     ]
    }
   ],
   "source": [
    "tos_max = dset['tos'].max('time', keep_attrs=True)\n",
    "print(tos_max)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hourly-moldova",
   "metadata": {},
   "source": [
    "It seems like the calculation happened instataneously, but it's actually just another \"lazy\" feature of `xarray`. It's showing us what the output of the calculation would look like (i.e. a 300 by 360 array), but `xarray` won't actually do the computation until the data is actually needed (e.g. to create a plot or write to a netCDF file). \n",
    "\n",
    "To force `xarray` to do the computation we can use `.compute()` with the `%%time` jupyter notebook command to record how long it takes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "twenty-peter",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/z3526123/opt/anaconda3/envs/pyaos-lesson/lib/python3.8/site-packages/dask/utils.py:30: RuntimeWarning: All-NaN slice encountered\n",
      "  return func(*args, **kwargs)\n",
      "/Users/z3526123/opt/anaconda3/envs/pyaos-lesson/lib/python3.8/site-packages/dask/core.py:121: RuntimeWarning: All-NaN slice encountered\n",
      "  return func(*(_execute_task(a, cache) for a in args))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2min 6s, sys: 2min 10s, total: 4min 17s\n",
      "Wall time: 1h 12min 20s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><svg style=\"position: absolute; width: 0; height: 0; overflow: hidden\">\n",
       "<defs>\n",
       "<symbol id=\"icon-database\" viewBox=\"0 0 32 32\">\n",
       "<path d=\"M16 0c-8.837 0-16 2.239-16 5v4c0 2.761 7.163 5 16 5s16-2.239 16-5v-4c0-2.761-7.163-5-16-5z\"></path>\n",
       "<path d=\"M16 17c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z\"></path>\n",
       "<path d=\"M16 26c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z\"></path>\n",
       "</symbol>\n",
       "<symbol id=\"icon-file-text2\" viewBox=\"0 0 32 32\">\n",
       "<path d=\"M28.681 7.159c-0.694-0.947-1.662-2.053-2.724-3.116s-2.169-2.030-3.116-2.724c-1.612-1.182-2.393-1.319-2.841-1.319h-15.5c-1.378 0-2.5 1.121-2.5 2.5v27c0 1.378 1.122 2.5 2.5 2.5h23c1.378 0 2.5-1.122 2.5-2.5v-19.5c0-0.448-0.137-1.23-1.319-2.841zM24.543 5.457c0.959 0.959 1.712 1.825 2.268 2.543h-4.811v-4.811c0.718 0.556 1.584 1.309 2.543 2.268zM28 29.5c0 0.271-0.229 0.5-0.5 0.5h-23c-0.271 0-0.5-0.229-0.5-0.5v-27c0-0.271 0.229-0.5 0.5-0.5 0 0 15.499-0 15.5 0v7c0 0.552 0.448 1 1 1h7v19.5z\"></path>\n",
       "<path d=\"M23 26h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
       "<path d=\"M23 22h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
       "<path d=\"M23 18h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
       "</symbol>\n",
       "</defs>\n",
       "</svg>\n",
       "<style>/* CSS stylesheet for displaying xarray objects in jupyterlab.\n",
       " *\n",
       " */\n",
       "\n",
       ":root {\n",
       "  --xr-font-color0: var(--jp-content-font-color0, rgba(0, 0, 0, 1));\n",
       "  --xr-font-color2: var(--jp-content-font-color2, rgba(0, 0, 0, 0.54));\n",
       "  --xr-font-color3: var(--jp-content-font-color3, rgba(0, 0, 0, 0.38));\n",
       "  --xr-border-color: var(--jp-border-color2, #e0e0e0);\n",
       "  --xr-disabled-color: var(--jp-layout-color3, #bdbdbd);\n",
       "  --xr-background-color: var(--jp-layout-color0, white);\n",
       "  --xr-background-color-row-even: var(--jp-layout-color1, white);\n",
       "  --xr-background-color-row-odd: var(--jp-layout-color2, #eeeeee);\n",
       "}\n",
       "\n",
       "html[theme=dark],\n",
       "body.vscode-dark {\n",
       "  --xr-font-color0: rgba(255, 255, 255, 1);\n",
       "  --xr-font-color2: rgba(255, 255, 255, 0.54);\n",
       "  --xr-font-color3: rgba(255, 255, 255, 0.38);\n",
       "  --xr-border-color: #1F1F1F;\n",
       "  --xr-disabled-color: #515151;\n",
       "  --xr-background-color: #111111;\n",
       "  --xr-background-color-row-even: #111111;\n",
       "  --xr-background-color-row-odd: #313131;\n",
       "}\n",
       "\n",
       ".xr-wrap {\n",
       "  display: block;\n",
       "  min-width: 300px;\n",
       "  max-width: 700px;\n",
       "}\n",
       "\n",
       ".xr-text-repr-fallback {\n",
       "  /* fallback to plain text repr when CSS is not injected (untrusted notebook) */\n",
       "  display: none;\n",
       "}\n",
       "\n",
       ".xr-header {\n",
       "  padding-top: 6px;\n",
       "  padding-bottom: 6px;\n",
       "  margin-bottom: 4px;\n",
       "  border-bottom: solid 1px var(--xr-border-color);\n",
       "}\n",
       "\n",
       ".xr-header > div,\n",
       ".xr-header > ul {\n",
       "  display: inline;\n",
       "  margin-top: 0;\n",
       "  margin-bottom: 0;\n",
       "}\n",
       "\n",
       ".xr-obj-type,\n",
       ".xr-array-name {\n",
       "  margin-left: 2px;\n",
       "  margin-right: 10px;\n",
       "}\n",
       "\n",
       ".xr-obj-type {\n",
       "  color: var(--xr-font-color2);\n",
       "}\n",
       "\n",
       ".xr-sections {\n",
       "  padding-left: 0 !important;\n",
       "  display: grid;\n",
       "  grid-template-columns: 150px auto auto 1fr 20px 20px;\n",
       "}\n",
       "\n",
       ".xr-section-item {\n",
       "  display: contents;\n",
       "}\n",
       "\n",
       ".xr-section-item input {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       ".xr-section-item input + label {\n",
       "  color: var(--xr-disabled-color);\n",
       "}\n",
       "\n",
       ".xr-section-item input:enabled + label {\n",
       "  cursor: pointer;\n",
       "  color: var(--xr-font-color2);\n",
       "}\n",
       "\n",
       ".xr-section-item input:enabled + label:hover {\n",
       "  color: var(--xr-font-color0);\n",
       "}\n",
       "\n",
       ".xr-section-summary {\n",
       "  grid-column: 1;\n",
       "  color: var(--xr-font-color2);\n",
       "  font-weight: 500;\n",
       "}\n",
       "\n",
       ".xr-section-summary > span {\n",
       "  display: inline-block;\n",
       "  padding-left: 0.5em;\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:disabled + label {\n",
       "  color: var(--xr-font-color2);\n",
       "}\n",
       "\n",
       ".xr-section-summary-in + label:before {\n",
       "  display: inline-block;\n",
       "  content: '►';\n",
       "  font-size: 11px;\n",
       "  width: 15px;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:disabled + label:before {\n",
       "  color: var(--xr-disabled-color);\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:checked + label:before {\n",
       "  content: '▼';\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:checked + label > span {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       ".xr-section-summary,\n",
       ".xr-section-inline-details {\n",
       "  padding-top: 4px;\n",
       "  padding-bottom: 4px;\n",
       "}\n",
       "\n",
       ".xr-section-inline-details {\n",
       "  grid-column: 2 / -1;\n",
       "}\n",
       "\n",
       ".xr-section-details {\n",
       "  display: none;\n",
       "  grid-column: 1 / -1;\n",
       "  margin-bottom: 5px;\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:checked ~ .xr-section-details {\n",
       "  display: contents;\n",
       "}\n",
       "\n",
       ".xr-array-wrap {\n",
       "  grid-column: 1 / -1;\n",
       "  display: grid;\n",
       "  grid-template-columns: 20px auto;\n",
       "}\n",
       "\n",
       ".xr-array-wrap > label {\n",
       "  grid-column: 1;\n",
       "  vertical-align: top;\n",
       "}\n",
       "\n",
       ".xr-preview {\n",
       "  color: var(--xr-font-color3);\n",
       "}\n",
       "\n",
       ".xr-array-preview,\n",
       ".xr-array-data {\n",
       "  padding: 0 5px !important;\n",
       "  grid-column: 2;\n",
       "}\n",
       "\n",
       ".xr-array-data,\n",
       ".xr-array-in:checked ~ .xr-array-preview {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       ".xr-array-in:checked ~ .xr-array-data,\n",
       ".xr-array-preview {\n",
       "  display: inline-block;\n",
       "}\n",
       "\n",
       ".xr-dim-list {\n",
       "  display: inline-block !important;\n",
       "  list-style: none;\n",
       "  padding: 0 !important;\n",
       "  margin: 0;\n",
       "}\n",
       "\n",
       ".xr-dim-list li {\n",
       "  display: inline-block;\n",
       "  padding: 0;\n",
       "  margin: 0;\n",
       "}\n",
       "\n",
       ".xr-dim-list:before {\n",
       "  content: '(';\n",
       "}\n",
       "\n",
       ".xr-dim-list:after {\n",
       "  content: ')';\n",
       "}\n",
       "\n",
       ".xr-dim-list li:not(:last-child):after {\n",
       "  content: ',';\n",
       "  padding-right: 5px;\n",
       "}\n",
       "\n",
       ".xr-has-index {\n",
       "  font-weight: bold;\n",
       "}\n",
       "\n",
       ".xr-var-list,\n",
       ".xr-var-item {\n",
       "  display: contents;\n",
       "}\n",
       "\n",
       ".xr-var-item > div,\n",
       ".xr-var-item label,\n",
       ".xr-var-item > .xr-var-name span {\n",
       "  background-color: var(--xr-background-color-row-even);\n",
       "  margin-bottom: 0;\n",
       "}\n",
       "\n",
       ".xr-var-item > .xr-var-name:hover span {\n",
       "  padding-right: 5px;\n",
       "}\n",
       "\n",
       ".xr-var-list > li:nth-child(odd) > div,\n",
       ".xr-var-list > li:nth-child(odd) > label,\n",
       ".xr-var-list > li:nth-child(odd) > .xr-var-name span {\n",
       "  background-color: var(--xr-background-color-row-odd);\n",
       "}\n",
       "\n",
       ".xr-var-name {\n",
       "  grid-column: 1;\n",
       "}\n",
       "\n",
       ".xr-var-dims {\n",
       "  grid-column: 2;\n",
       "}\n",
       "\n",
       ".xr-var-dtype {\n",
       "  grid-column: 3;\n",
       "  text-align: right;\n",
       "  color: var(--xr-font-color2);\n",
       "}\n",
       "\n",
       ".xr-var-preview {\n",
       "  grid-column: 4;\n",
       "}\n",
       "\n",
       ".xr-var-name,\n",
       ".xr-var-dims,\n",
       ".xr-var-dtype,\n",
       ".xr-preview,\n",
       ".xr-attrs dt {\n",
       "  white-space: nowrap;\n",
       "  overflow: hidden;\n",
       "  text-overflow: ellipsis;\n",
       "  padding-right: 10px;\n",
       "}\n",
       "\n",
       ".xr-var-name:hover,\n",
       ".xr-var-dims:hover,\n",
       ".xr-var-dtype:hover,\n",
       ".xr-attrs dt:hover {\n",
       "  overflow: visible;\n",
       "  width: auto;\n",
       "  z-index: 1;\n",
       "}\n",
       "\n",
       ".xr-var-attrs,\n",
       ".xr-var-data {\n",
       "  display: none;\n",
       "  background-color: var(--xr-background-color) !important;\n",
       "  padding-bottom: 5px !important;\n",
       "}\n",
       "\n",
       ".xr-var-attrs-in:checked ~ .xr-var-attrs,\n",
       ".xr-var-data-in:checked ~ .xr-var-data {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       ".xr-var-data > table {\n",
       "  float: right;\n",
       "}\n",
       "\n",
       ".xr-var-name span,\n",
       ".xr-var-data,\n",
       ".xr-attrs {\n",
       "  padding-left: 25px !important;\n",
       "}\n",
       "\n",
       ".xr-attrs,\n",
       ".xr-var-attrs,\n",
       ".xr-var-data {\n",
       "  grid-column: 1 / -1;\n",
       "}\n",
       "\n",
       "dl.xr-attrs {\n",
       "  padding: 0;\n",
       "  margin: 0;\n",
       "  display: grid;\n",
       "  grid-template-columns: 125px auto;\n",
       "}\n",
       "\n",
       ".xr-attrs dt,\n",
       ".xr-attrs dd {\n",
       "  padding: 0;\n",
       "  margin: 0;\n",
       "  float: left;\n",
       "  padding-right: 10px;\n",
       "  width: auto;\n",
       "}\n",
       "\n",
       ".xr-attrs dt {\n",
       "  font-weight: normal;\n",
       "  grid-column: 1;\n",
       "}\n",
       "\n",
       ".xr-attrs dt:hover span {\n",
       "  display: inline-block;\n",
       "  background: var(--xr-background-color);\n",
       "  padding-right: 10px;\n",
       "}\n",
       "\n",
       ".xr-attrs dd {\n",
       "  grid-column: 2;\n",
       "  white-space: pre-wrap;\n",
       "  word-break: break-all;\n",
       "}\n",
       "\n",
       ".xr-icon-database,\n",
       ".xr-icon-file-text2 {\n",
       "  display: inline-block;\n",
       "  vertical-align: middle;\n",
       "  width: 1em;\n",
       "  height: 1.5em !important;\n",
       "  stroke-width: 0;\n",
       "  stroke: currentColor;\n",
       "  fill: currentColor;\n",
       "}\n",
       "</style><pre class='xr-text-repr-fallback'>&lt;xarray.DataArray &#x27;tos&#x27; (j: 300, i: 360)&gt;\n",
       "array([[ 2.81750893e+20,  2.07011706e-19,  1.35559949e-19, ...,\n",
       "        -1.27854324e-11, -1.27854324e-11, -1.27854324e-11],\n",
       "       [-1.27854324e-11, -1.27854324e-11, -1.27854324e-11, ...,\n",
       "        -1.27854324e-11, -1.27854324e-11, -1.27854324e-11],\n",
       "       [-1.27854324e-11, -1.27854324e-11, -1.27854324e-11, ...,\n",
       "        -1.27854324e-11, -1.27854324e-11, -1.27854324e-11],\n",
       "       ...,\n",
       "       [-6.72963876e+15, -6.74107895e+15, -2.68975011e+16, ...,\n",
       "        -4.20162121e+14, -1.68172840e+15, -6.73270967e+15],\n",
       "       [-6.74393295e+15, -6.73430417e+15, -1.07628419e+17, ...,\n",
       "        -4.20084678e+14, -1.68490091e+15, -6.74294135e+15],\n",
       "       [-6.74568476e+15, -6.72096400e+15, -1.07771286e+17, ...,\n",
       "        -1.05425324e+14, -1.68527631e+15, -6.72055276e+15]], dtype=float32)\n",
       "Coordinates:\n",
       "  * j          (j) int32 0 1 2 3 4 5 6 7 8 ... 292 293 294 295 296 297 298 299\n",
       "  * i          (i) int32 0 1 2 3 4 5 6 7 8 ... 352 353 354 355 356 357 358 359\n",
       "    latitude   (j, i) float64 -77.88 -77.88 -77.88 -77.88 ... 66.04 65.63 65.21\n",
       "    longitude  (j, i) float64 80.5 81.5 82.5 83.5 ... 79.94 79.96 79.97 79.99\n",
       "Attributes:\n",
       "    standard_name:  sea_surface_temperature\n",
       "    long_name:      Sea Surface Temperature\n",
       "    comment:        Temperature of upper boundary of the liquid ocean, includ...\n",
       "    units:          degC\n",
       "    cell_methods:   area: mean where sea time: mean\n",
       "    cell_measures:  area: areacello\n",
       "    history:        2019-11-08T18:23:54Z altered by CMOR: replaced missing va...\n",
       "    _ChunkSizes:    [  1 300 360]</pre><div class='xr-wrap' hidden><div class='xr-header'><div class='xr-obj-type'>xarray.DataArray</div><div class='xr-array-name'>'tos'</div><ul class='xr-dim-list'><li><span class='xr-has-index'>j</span>: 300</li><li><span class='xr-has-index'>i</span>: 360</li></ul></div><ul class='xr-sections'><li class='xr-section-item'><div class='xr-array-wrap'><input id='section-217c6cca-1ff8-40ae-97dc-1377c4dca0ca' class='xr-array-in' type='checkbox' checked><label for='section-217c6cca-1ff8-40ae-97dc-1377c4dca0ca' title='Show/hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-array-preview xr-preview'><span>2.818e+20 2.07e-19 1.356e-19 ... -1.054e+14 -1.685e+15 -6.721e+15</span></div><div class='xr-array-data'><pre>array([[ 2.81750893e+20,  2.07011706e-19,  1.35559949e-19, ...,\n",
       "        -1.27854324e-11, -1.27854324e-11, -1.27854324e-11],\n",
       "       [-1.27854324e-11, -1.27854324e-11, -1.27854324e-11, ...,\n",
       "        -1.27854324e-11, -1.27854324e-11, -1.27854324e-11],\n",
       "       [-1.27854324e-11, -1.27854324e-11, -1.27854324e-11, ...,\n",
       "        -1.27854324e-11, -1.27854324e-11, -1.27854324e-11],\n",
       "       ...,\n",
       "       [-6.72963876e+15, -6.74107895e+15, -2.68975011e+16, ...,\n",
       "        -4.20162121e+14, -1.68172840e+15, -6.73270967e+15],\n",
       "       [-6.74393295e+15, -6.73430417e+15, -1.07628419e+17, ...,\n",
       "        -4.20084678e+14, -1.68490091e+15, -6.74294135e+15],\n",
       "       [-6.74568476e+15, -6.72096400e+15, -1.07771286e+17, ...,\n",
       "        -1.05425324e+14, -1.68527631e+15, -6.72055276e+15]], dtype=float32)</pre></div></div></li><li class='xr-section-item'><input id='section-7a259b03-3ff9-4e33-891b-14ca612529e0' class='xr-section-summary-in' type='checkbox'  checked><label for='section-7a259b03-3ff9-4e33-891b-14ca612529e0' class='xr-section-summary' >Coordinates: <span>(4)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>j</span></div><div class='xr-var-dims'>(j)</div><div class='xr-var-dtype'>int32</div><div class='xr-var-preview xr-preview'>0 1 2 3 4 5 ... 295 296 297 298 299</div><input id='attrs-1a45b964-545b-4661-a660-856245169fd3' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-1a45b964-545b-4661-a660-856245169fd3' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-d2920e46-8009-4cf5-b602-08b51335a96f' class='xr-var-data-in' type='checkbox'><label for='data-d2920e46-8009-4cf5-b602-08b51335a96f' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>units :</span></dt><dd>1</dd><dt><span>long_name :</span></dt><dd>cell index along second dimension</dd></dl></div><div class='xr-var-data'><pre>array([  0,   1,   2, ..., 297, 298, 299], dtype=int32)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>i</span></div><div class='xr-var-dims'>(i)</div><div class='xr-var-dtype'>int32</div><div class='xr-var-preview xr-preview'>0 1 2 3 4 5 ... 355 356 357 358 359</div><input id='attrs-5e92028b-9b07-4137-8c85-2b2228282f07' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-5e92028b-9b07-4137-8c85-2b2228282f07' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-e16ce56d-b2b9-4972-b30b-7b9051ff7fd6' class='xr-var-data-in' type='checkbox'><label for='data-e16ce56d-b2b9-4972-b30b-7b9051ff7fd6' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>units :</span></dt><dd>1</dd><dt><span>long_name :</span></dt><dd>cell index along first dimension</dd></dl></div><div class='xr-var-data'><pre>array([  0,   1,   2, ..., 357, 358, 359], dtype=int32)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>latitude</span></div><div class='xr-var-dims'>(j, i)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>-77.88 -77.88 ... 65.63 65.21</div><input id='attrs-6a5b389b-e304-4568-be3f-6e8c49be9c0e' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-6a5b389b-e304-4568-be3f-6e8c49be9c0e' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-9d6d49b1-bcb3-492c-af7f-344c29ee0480' class='xr-var-data-in' type='checkbox'><label for='data-9d6d49b1-bcb3-492c-af7f-344c29ee0480' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>standard_name :</span></dt><dd>latitude</dd><dt><span>long_name :</span></dt><dd>latitude</dd><dt><span>units :</span></dt><dd>degrees_north</dd><dt><span>bounds :</span></dt><dd>vertices_latitude</dd><dt><span>_ChunkSizes :</span></dt><dd>[300 360]</dd></dl></div><div class='xr-var-data'><pre>array([[-77.87662338, -77.87662338, -77.87662338, ..., -77.87662338,\n",
       "        -77.87662338, -77.87662338],\n",
       "       [-77.6297134 , -77.6297134 , -77.6297134 , ..., -77.6297134 ,\n",
       "        -77.6297134 , -77.6297134 ],\n",
       "       [-77.38170671, -77.38170671, -77.38170671, ..., -77.38170671,\n",
       "        -77.38170671, -77.38170671],\n",
       "       ...,\n",
       "       [ 65.20965413,  65.62409039,  66.03219689, ...,  66.03219689,\n",
       "         65.62409039,  65.20965413],\n",
       "       [ 65.21018119,  65.62564754,  66.0347538 , ...,  66.0347538 ,\n",
       "         65.62564754,  65.21018119],\n",
       "       [ 65.21044485,  65.62642645,  66.03603278, ...,  66.03603278,\n",
       "         65.62642645,  65.21044485]])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>longitude</span></div><div class='xr-var-dims'>(j, i)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>80.5 81.5 82.5 ... 79.97 79.99</div><input id='attrs-f2b04919-eccb-4f66-a0dc-05e7fcbd2bd9' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-f2b04919-eccb-4f66-a0dc-05e7fcbd2bd9' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-40c5d096-438b-4075-b69a-2380d26873a3' class='xr-var-data-in' type='checkbox'><label for='data-40c5d096-438b-4075-b69a-2380d26873a3' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>standard_name :</span></dt><dd>longitude</dd><dt><span>long_name :</span></dt><dd>longitude</dd><dt><span>units :</span></dt><dd>degrees_east</dd><dt><span>bounds :</span></dt><dd>vertices_longitude</dd><dt><span>_ChunkSizes :</span></dt><dd>[300 360]</dd></dl></div><div class='xr-var-data'><pre>array([[80.5       , 81.5       , 82.5       , ..., 77.5       ,\n",
       "        78.5       , 79.5       ],\n",
       "       [80.5       , 81.5       , 82.5       , ..., 77.5       ,\n",
       "        78.5       , 79.5       ],\n",
       "       [80.5       , 81.5       , 82.5       , ..., 77.5       ,\n",
       "        78.5       , 79.5       ],\n",
       "       ...,\n",
       "       [80.04428052, 80.13286833, 80.22153653, ..., 79.77846347,\n",
       "        79.86713167, 79.95571948],\n",
       "       [80.02659156, 80.07979084, 80.13303864, ..., 79.86696136,\n",
       "        79.92020916, 79.97340844],\n",
       "       [80.00887691, 80.02663615, 80.04441162, ..., 79.95558838,\n",
       "        79.97336385, 79.99112309]])</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-cee65227-4678-4669-a6e4-8f4ed52141bb' class='xr-section-summary-in' type='checkbox'  checked><label for='section-cee65227-4678-4669-a6e4-8f4ed52141bb' class='xr-section-summary' >Attributes: <span>(8)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><dl class='xr-attrs'><dt><span>standard_name :</span></dt><dd>sea_surface_temperature</dd><dt><span>long_name :</span></dt><dd>Sea Surface Temperature</dd><dt><span>comment :</span></dt><dd>Temperature of upper boundary of the liquid ocean, including temperatures below sea-ice and floating ice shelves.</dd><dt><span>units :</span></dt><dd>degC</dd><dt><span>cell_methods :</span></dt><dd>area: mean where sea time: mean</dd><dt><span>cell_measures :</span></dt><dd>area: areacello</dd><dt><span>history :</span></dt><dd>2019-11-08T18:23:54Z altered by CMOR: replaced missing value flag (-1e+20) with standard missing value (1e+20).</dd><dt><span>_ChunkSizes :</span></dt><dd>[  1 300 360]</dd></dl></div></li></ul></div></div>"
      ],
      "text/plain": [
       "<xarray.DataArray 'tos' (j: 300, i: 360)>\n",
       "array([[ 2.81750893e+20,  2.07011706e-19,  1.35559949e-19, ...,\n",
       "        -1.27854324e-11, -1.27854324e-11, -1.27854324e-11],\n",
       "       [-1.27854324e-11, -1.27854324e-11, -1.27854324e-11, ...,\n",
       "        -1.27854324e-11, -1.27854324e-11, -1.27854324e-11],\n",
       "       [-1.27854324e-11, -1.27854324e-11, -1.27854324e-11, ...,\n",
       "        -1.27854324e-11, -1.27854324e-11, -1.27854324e-11],\n",
       "       ...,\n",
       "       [-6.72963876e+15, -6.74107895e+15, -2.68975011e+16, ...,\n",
       "        -4.20162121e+14, -1.68172840e+15, -6.73270967e+15],\n",
       "       [-6.74393295e+15, -6.73430417e+15, -1.07628419e+17, ...,\n",
       "        -4.20084678e+14, -1.68490091e+15, -6.74294135e+15],\n",
       "       [-6.74568476e+15, -6.72096400e+15, -1.07771286e+17, ...,\n",
       "        -1.05425324e+14, -1.68527631e+15, -6.72055276e+15]], dtype=float32)\n",
       "Coordinates:\n",
       "  * j          (j) int32 0 1 2 3 4 5 6 7 8 ... 292 293 294 295 296 297 298 299\n",
       "  * i          (i) int32 0 1 2 3 4 5 6 7 8 ... 352 353 354 355 356 357 358 359\n",
       "    latitude   (j, i) float64 -77.88 -77.88 -77.88 -77.88 ... 66.04 65.63 65.21\n",
       "    longitude  (j, i) float64 80.5 81.5 82.5 83.5 ... 79.94 79.96 79.97 79.99\n",
       "Attributes:\n",
       "    standard_name:  sea_surface_temperature\n",
       "    long_name:      Sea Surface Temperature\n",
       "    comment:        Temperature of upper boundary of the liquid ocean, includ...\n",
       "    units:          degC\n",
       "    cell_methods:   area: mean where sea time: mean\n",
       "    cell_measures:  area: areacello\n",
       "    history:        2019-11-08T18:23:54Z altered by CMOR: replaced missing va...\n",
       "    _ChunkSizes:    [  1 300 360]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "tos_max.compute()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "revolutionary-generic",
   "metadata": {},
   "source": [
    "It took a little over an hour to compute the daily maximum SST (this might be different on your laptop). As shown below, the data array (26GB) we are working with is larger than the memory available on our laptop (~17GB), so by processing the data one (231, 300, 36) chunk at a time we avoided causing a memory error. In other words, the calculation wouldn't have been possible without chunking."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "equipped-depth",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<tr>\n",
       "<td>\n",
       "<table>\n",
       "  <thead>\n",
       "    <tr><td> </td><th> Array </th><th> Chunk </th></tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr><th> Bytes </th><td> 26.03 GB </td> <td> 99.79 MB </td></tr>\n",
       "    <tr><th> Shape </th><td> (60265, 300, 360) </td> <td> (231, 300, 360) </td></tr>\n",
       "    <tr><th> Count </th><td> 659 Tasks </td><td> 321 Chunks </td></tr>\n",
       "    <tr><th> Type </th><td> float32 </td><td> numpy.ndarray </td></tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</td>\n",
       "<td>\n",
       "<svg width=\"156\" height=\"146\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"10\" y1=\"0\" x2=\"80\" y2=\"70\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"10\" y1=\"25\" x2=\"80\" y2=\"96\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"10\" y1=\"0\" x2=\"10\" y2=\"25\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"13\" y1=\"3\" x2=\"13\" y2=\"28\" />\n",
       "  <line x1=\"17\" y1=\"7\" x2=\"17\" y2=\"32\" />\n",
       "  <line x1=\"20\" y1=\"10\" x2=\"20\" y2=\"36\" />\n",
       "  <line x1=\"24\" y1=\"14\" x2=\"24\" y2=\"40\" />\n",
       "  <line x1=\"28\" y1=\"18\" x2=\"28\" y2=\"44\" />\n",
       "  <line x1=\"31\" y1=\"21\" x2=\"31\" y2=\"47\" />\n",
       "  <line x1=\"35\" y1=\"25\" x2=\"35\" y2=\"50\" />\n",
       "  <line x1=\"38\" y1=\"28\" x2=\"38\" y2=\"53\" />\n",
       "  <line x1=\"42\" y1=\"32\" x2=\"42\" y2=\"58\" />\n",
       "  <line x1=\"46\" y1=\"36\" x2=\"46\" y2=\"61\" />\n",
       "  <line x1=\"49\" y1=\"39\" x2=\"49\" y2=\"65\" />\n",
       "  <line x1=\"53\" y1=\"43\" x2=\"53\" y2=\"69\" />\n",
       "  <line x1=\"57\" y1=\"47\" x2=\"57\" y2=\"72\" />\n",
       "  <line x1=\"61\" y1=\"51\" x2=\"61\" y2=\"77\" />\n",
       "  <line x1=\"65\" y1=\"55\" x2=\"65\" y2=\"80\" />\n",
       "  <line x1=\"69\" y1=\"59\" x2=\"69\" y2=\"84\" />\n",
       "  <line x1=\"72\" y1=\"62\" x2=\"72\" y2=\"88\" />\n",
       "  <line x1=\"76\" y1=\"66\" x2=\"76\" y2=\"92\" />\n",
       "  <line x1=\"80\" y1=\"70\" x2=\"80\" y2=\"96\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"10.0,0.0 80.58823529411765,70.58823529411765 80.58823529411765,96.00085180870013 10.0,25.412616514582485\" style=\"fill:#8B4903A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"10\" y1=\"0\" x2=\"35\" y2=\"0\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"13\" y1=\"3\" x2=\"38\" y2=\"3\" />\n",
       "  <line x1=\"17\" y1=\"7\" x2=\"42\" y2=\"7\" />\n",
       "  <line x1=\"20\" y1=\"10\" x2=\"46\" y2=\"10\" />\n",
       "  <line x1=\"24\" y1=\"14\" x2=\"50\" y2=\"14\" />\n",
       "  <line x1=\"28\" y1=\"18\" x2=\"54\" y2=\"18\" />\n",
       "  <line x1=\"31\" y1=\"21\" x2=\"57\" y2=\"21\" />\n",
       "  <line x1=\"35\" y1=\"25\" x2=\"60\" y2=\"25\" />\n",
       "  <line x1=\"38\" y1=\"28\" x2=\"63\" y2=\"28\" />\n",
       "  <line x1=\"42\" y1=\"32\" x2=\"68\" y2=\"32\" />\n",
       "  <line x1=\"46\" y1=\"36\" x2=\"71\" y2=\"36\" />\n",
       "  <line x1=\"49\" y1=\"39\" x2=\"75\" y2=\"39\" />\n",
       "  <line x1=\"53\" y1=\"43\" x2=\"79\" y2=\"43\" />\n",
       "  <line x1=\"57\" y1=\"47\" x2=\"82\" y2=\"47\" />\n",
       "  <line x1=\"61\" y1=\"51\" x2=\"87\" y2=\"51\" />\n",
       "  <line x1=\"65\" y1=\"55\" x2=\"90\" y2=\"55\" />\n",
       "  <line x1=\"69\" y1=\"59\" x2=\"94\" y2=\"59\" />\n",
       "  <line x1=\"72\" y1=\"62\" x2=\"98\" y2=\"62\" />\n",
       "  <line x1=\"76\" y1=\"66\" x2=\"102\" y2=\"66\" />\n",
       "  <line x1=\"80\" y1=\"70\" x2=\"106\" y2=\"70\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"10\" y1=\"0\" x2=\"80\" y2=\"70\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"35\" y1=\"0\" x2=\"106\" y2=\"70\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"10.0,0.0 35.41261651458249,0.0 106.00085180870013,70.58823529411765 80.58823529411765,70.58823529411765\" style=\"fill:#8B4903A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"80\" y1=\"70\" x2=\"106\" y2=\"70\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"80\" y1=\"96\" x2=\"106\" y2=\"96\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"80\" y1=\"70\" x2=\"80\" y2=\"96\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"106\" y1=\"70\" x2=\"106\" y2=\"96\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"80.58823529411765,70.58823529411765 106.00085180870013,70.58823529411765 106.00085180870013,96.00085180870013 80.58823529411765,96.00085180870013\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Text -->\n",
       "  <text x=\"93.294544\" y=\"116.000852\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >360</text>\n",
       "  <text x=\"126.000852\" y=\"83.294544\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(-90,126.000852,83.294544)\">300</text>\n",
       "  <text x=\"35.294118\" y=\"80.706734\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(45,35.294118,80.706734)\">60265</text>\n",
       "</svg>\n",
       "</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "dask.array<concatenate, shape=(60265, 300, 360), dtype=float32, chunksize=(231, 300, 360), chunktype=numpy.ndarray>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dset['tos'].data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "extensive-completion",
   "metadata": {},
   "source": [
    "Part of the reason the calculation took over an hour is that it was all done on one core. We can try and speed things up by using a `dask` \"client\" to run the calculation in parallel across multiple cores:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "exciting-kingdom",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table style=\"border: 2px solid white;\">\n",
       "<tr>\n",
       "<td style=\"vertical-align: top; border: 0px solid white\">\n",
       "<h3 style=\"text-align: left;\">Client</h3>\n",
       "<ul style=\"text-align: left; list-style: none; margin: 0; padding: 0;\">\n",
       "  <li><b>Scheduler: </b>tcp://127.0.0.1:60760</li>\n",
       "  <li><b>Dashboard: </b><a href='http://127.0.0.1:8787/status' target='_blank'>http://127.0.0.1:8787/status</a></li>\n",
       "</ul>\n",
       "</td>\n",
       "<td style=\"vertical-align: top; border: 0px solid white\">\n",
       "<h3 style=\"text-align: left;\">Cluster</h3>\n",
       "<ul style=\"text-align: left; list-style:none; margin: 0; padding: 0;\">\n",
       "  <li><b>Workers: </b>4</li>\n",
       "  <li><b>Cores: </b>4</li>\n",
       "  <li><b>Memory: </b>17.18 GB</li>\n",
       "</ul>\n",
       "</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<Client: 'tcp://127.0.0.1:60760' processes=4 threads=4, memory=17.18 GB>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from dask.distributed import Client\n",
    "client = Client()\n",
    "client"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "numeric-transportation",
   "metadata": {},
   "source": [
    "(Click on the dashboard link to watch what's happening on each core.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "meaningful-dominant",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2min 43s, sys: 22.6 s, total: 3min 5s\n",
      "Wall time: 1h 6min 47s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "maximum = tos_max.compute()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "independent-pocket",
   "metadata": {},
   "source": [
    "FIXME: The values are all nan.\n",
    "\n",
    "By distributing the calculation across all four cores the processing time has dropped to 24 minutes. This is close to but not quite a quarter of the original 1 hour and 12 minutes, because there's a time cost associated with setting up and coordinating jobs across all the cores.\n",
    "\n",
    "> **Parallel isn't always best**\n",
    ">\n",
    "> For smaller or very complex data processing tasks,\n",
    "> the time associated with setting up and coordinating jobs across multiple cores\n",
    "> can mean it takes longer to run in parallel than on just one core."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "overhead-saver",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><svg style=\"position: absolute; width: 0; height: 0; overflow: hidden\">\n",
       "<defs>\n",
       "<symbol id=\"icon-database\" viewBox=\"0 0 32 32\">\n",
       "<path d=\"M16 0c-8.837 0-16 2.239-16 5v4c0 2.761 7.163 5 16 5s16-2.239 16-5v-4c0-2.761-7.163-5-16-5z\"></path>\n",
       "<path d=\"M16 17c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z\"></path>\n",
       "<path d=\"M16 26c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z\"></path>\n",
       "</symbol>\n",
       "<symbol id=\"icon-file-text2\" viewBox=\"0 0 32 32\">\n",
       "<path d=\"M28.681 7.159c-0.694-0.947-1.662-2.053-2.724-3.116s-2.169-2.030-3.116-2.724c-1.612-1.182-2.393-1.319-2.841-1.319h-15.5c-1.378 0-2.5 1.121-2.5 2.5v27c0 1.378 1.122 2.5 2.5 2.5h23c1.378 0 2.5-1.122 2.5-2.5v-19.5c0-0.448-0.137-1.23-1.319-2.841zM24.543 5.457c0.959 0.959 1.712 1.825 2.268 2.543h-4.811v-4.811c0.718 0.556 1.584 1.309 2.543 2.268zM28 29.5c0 0.271-0.229 0.5-0.5 0.5h-23c-0.271 0-0.5-0.229-0.5-0.5v-27c0-0.271 0.229-0.5 0.5-0.5 0 0 15.499-0 15.5 0v7c0 0.552 0.448 1 1 1h7v19.5z\"></path>\n",
       "<path d=\"M23 26h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
       "<path d=\"M23 22h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
       "<path d=\"M23 18h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
       "</symbol>\n",
       "</defs>\n",
       "</svg>\n",
       "<style>/* CSS stylesheet for displaying xarray objects in jupyterlab.\n",
       " *\n",
       " */\n",
       "\n",
       ":root {\n",
       "  --xr-font-color0: var(--jp-content-font-color0, rgba(0, 0, 0, 1));\n",
       "  --xr-font-color2: var(--jp-content-font-color2, rgba(0, 0, 0, 0.54));\n",
       "  --xr-font-color3: var(--jp-content-font-color3, rgba(0, 0, 0, 0.38));\n",
       "  --xr-border-color: var(--jp-border-color2, #e0e0e0);\n",
       "  --xr-disabled-color: var(--jp-layout-color3, #bdbdbd);\n",
       "  --xr-background-color: var(--jp-layout-color0, white);\n",
       "  --xr-background-color-row-even: var(--jp-layout-color1, white);\n",
       "  --xr-background-color-row-odd: var(--jp-layout-color2, #eeeeee);\n",
       "}\n",
       "\n",
       "html[theme=dark],\n",
       "body.vscode-dark {\n",
       "  --xr-font-color0: rgba(255, 255, 255, 1);\n",
       "  --xr-font-color2: rgba(255, 255, 255, 0.54);\n",
       "  --xr-font-color3: rgba(255, 255, 255, 0.38);\n",
       "  --xr-border-color: #1F1F1F;\n",
       "  --xr-disabled-color: #515151;\n",
       "  --xr-background-color: #111111;\n",
       "  --xr-background-color-row-even: #111111;\n",
       "  --xr-background-color-row-odd: #313131;\n",
       "}\n",
       "\n",
       ".xr-wrap {\n",
       "  display: block;\n",
       "  min-width: 300px;\n",
       "  max-width: 700px;\n",
       "}\n",
       "\n",
       ".xr-text-repr-fallback {\n",
       "  /* fallback to plain text repr when CSS is not injected (untrusted notebook) */\n",
       "  display: none;\n",
       "}\n",
       "\n",
       ".xr-header {\n",
       "  padding-top: 6px;\n",
       "  padding-bottom: 6px;\n",
       "  margin-bottom: 4px;\n",
       "  border-bottom: solid 1px var(--xr-border-color);\n",
       "}\n",
       "\n",
       ".xr-header > div,\n",
       ".xr-header > ul {\n",
       "  display: inline;\n",
       "  margin-top: 0;\n",
       "  margin-bottom: 0;\n",
       "}\n",
       "\n",
       ".xr-obj-type,\n",
       ".xr-array-name {\n",
       "  margin-left: 2px;\n",
       "  margin-right: 10px;\n",
       "}\n",
       "\n",
       ".xr-obj-type {\n",
       "  color: var(--xr-font-color2);\n",
       "}\n",
       "\n",
       ".xr-sections {\n",
       "  padding-left: 0 !important;\n",
       "  display: grid;\n",
       "  grid-template-columns: 150px auto auto 1fr 20px 20px;\n",
       "}\n",
       "\n",
       ".xr-section-item {\n",
       "  display: contents;\n",
       "}\n",
       "\n",
       ".xr-section-item input {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       ".xr-section-item input + label {\n",
       "  color: var(--xr-disabled-color);\n",
       "}\n",
       "\n",
       ".xr-section-item input:enabled + label {\n",
       "  cursor: pointer;\n",
       "  color: var(--xr-font-color2);\n",
       "}\n",
       "\n",
       ".xr-section-item input:enabled + label:hover {\n",
       "  color: var(--xr-font-color0);\n",
       "}\n",
       "\n",
       ".xr-section-summary {\n",
       "  grid-column: 1;\n",
       "  color: var(--xr-font-color2);\n",
       "  font-weight: 500;\n",
       "}\n",
       "\n",
       ".xr-section-summary > span {\n",
       "  display: inline-block;\n",
       "  padding-left: 0.5em;\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:disabled + label {\n",
       "  color: var(--xr-font-color2);\n",
       "}\n",
       "\n",
       ".xr-section-summary-in + label:before {\n",
       "  display: inline-block;\n",
       "  content: '►';\n",
       "  font-size: 11px;\n",
       "  width: 15px;\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:disabled + label:before {\n",
       "  color: var(--xr-disabled-color);\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:checked + label:before {\n",
       "  content: '▼';\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:checked + label > span {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       ".xr-section-summary,\n",
       ".xr-section-inline-details {\n",
       "  padding-top: 4px;\n",
       "  padding-bottom: 4px;\n",
       "}\n",
       "\n",
       ".xr-section-inline-details {\n",
       "  grid-column: 2 / -1;\n",
       "}\n",
       "\n",
       ".xr-section-details {\n",
       "  display: none;\n",
       "  grid-column: 1 / -1;\n",
       "  margin-bottom: 5px;\n",
       "}\n",
       "\n",
       ".xr-section-summary-in:checked ~ .xr-section-details {\n",
       "  display: contents;\n",
       "}\n",
       "\n",
       ".xr-array-wrap {\n",
       "  grid-column: 1 / -1;\n",
       "  display: grid;\n",
       "  grid-template-columns: 20px auto;\n",
       "}\n",
       "\n",
       ".xr-array-wrap > label {\n",
       "  grid-column: 1;\n",
       "  vertical-align: top;\n",
       "}\n",
       "\n",
       ".xr-preview {\n",
       "  color: var(--xr-font-color3);\n",
       "}\n",
       "\n",
       ".xr-array-preview,\n",
       ".xr-array-data {\n",
       "  padding: 0 5px !important;\n",
       "  grid-column: 2;\n",
       "}\n",
       "\n",
       ".xr-array-data,\n",
       ".xr-array-in:checked ~ .xr-array-preview {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       ".xr-array-in:checked ~ .xr-array-data,\n",
       ".xr-array-preview {\n",
       "  display: inline-block;\n",
       "}\n",
       "\n",
       ".xr-dim-list {\n",
       "  display: inline-block !important;\n",
       "  list-style: none;\n",
       "  padding: 0 !important;\n",
       "  margin: 0;\n",
       "}\n",
       "\n",
       ".xr-dim-list li {\n",
       "  display: inline-block;\n",
       "  padding: 0;\n",
       "  margin: 0;\n",
       "}\n",
       "\n",
       ".xr-dim-list:before {\n",
       "  content: '(';\n",
       "}\n",
       "\n",
       ".xr-dim-list:after {\n",
       "  content: ')';\n",
       "}\n",
       "\n",
       ".xr-dim-list li:not(:last-child):after {\n",
       "  content: ',';\n",
       "  padding-right: 5px;\n",
       "}\n",
       "\n",
       ".xr-has-index {\n",
       "  font-weight: bold;\n",
       "}\n",
       "\n",
       ".xr-var-list,\n",
       ".xr-var-item {\n",
       "  display: contents;\n",
       "}\n",
       "\n",
       ".xr-var-item > div,\n",
       ".xr-var-item label,\n",
       ".xr-var-item > .xr-var-name span {\n",
       "  background-color: var(--xr-background-color-row-even);\n",
       "  margin-bottom: 0;\n",
       "}\n",
       "\n",
       ".xr-var-item > .xr-var-name:hover span {\n",
       "  padding-right: 5px;\n",
       "}\n",
       "\n",
       ".xr-var-list > li:nth-child(odd) > div,\n",
       ".xr-var-list > li:nth-child(odd) > label,\n",
       ".xr-var-list > li:nth-child(odd) > .xr-var-name span {\n",
       "  background-color: var(--xr-background-color-row-odd);\n",
       "}\n",
       "\n",
       ".xr-var-name {\n",
       "  grid-column: 1;\n",
       "}\n",
       "\n",
       ".xr-var-dims {\n",
       "  grid-column: 2;\n",
       "}\n",
       "\n",
       ".xr-var-dtype {\n",
       "  grid-column: 3;\n",
       "  text-align: right;\n",
       "  color: var(--xr-font-color2);\n",
       "}\n",
       "\n",
       ".xr-var-preview {\n",
       "  grid-column: 4;\n",
       "}\n",
       "\n",
       ".xr-var-name,\n",
       ".xr-var-dims,\n",
       ".xr-var-dtype,\n",
       ".xr-preview,\n",
       ".xr-attrs dt {\n",
       "  white-space: nowrap;\n",
       "  overflow: hidden;\n",
       "  text-overflow: ellipsis;\n",
       "  padding-right: 10px;\n",
       "}\n",
       "\n",
       ".xr-var-name:hover,\n",
       ".xr-var-dims:hover,\n",
       ".xr-var-dtype:hover,\n",
       ".xr-attrs dt:hover {\n",
       "  overflow: visible;\n",
       "  width: auto;\n",
       "  z-index: 1;\n",
       "}\n",
       "\n",
       ".xr-var-attrs,\n",
       ".xr-var-data {\n",
       "  display: none;\n",
       "  background-color: var(--xr-background-color) !important;\n",
       "  padding-bottom: 5px !important;\n",
       "}\n",
       "\n",
       ".xr-var-attrs-in:checked ~ .xr-var-attrs,\n",
       ".xr-var-data-in:checked ~ .xr-var-data {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       ".xr-var-data > table {\n",
       "  float: right;\n",
       "}\n",
       "\n",
       ".xr-var-name span,\n",
       ".xr-var-data,\n",
       ".xr-attrs {\n",
       "  padding-left: 25px !important;\n",
       "}\n",
       "\n",
       ".xr-attrs,\n",
       ".xr-var-attrs,\n",
       ".xr-var-data {\n",
       "  grid-column: 1 / -1;\n",
       "}\n",
       "\n",
       "dl.xr-attrs {\n",
       "  padding: 0;\n",
       "  margin: 0;\n",
       "  display: grid;\n",
       "  grid-template-columns: 125px auto;\n",
       "}\n",
       "\n",
       ".xr-attrs dt,\n",
       ".xr-attrs dd {\n",
       "  padding: 0;\n",
       "  margin: 0;\n",
       "  float: left;\n",
       "  padding-right: 10px;\n",
       "  width: auto;\n",
       "}\n",
       "\n",
       ".xr-attrs dt {\n",
       "  font-weight: normal;\n",
       "  grid-column: 1;\n",
       "}\n",
       "\n",
       ".xr-attrs dt:hover span {\n",
       "  display: inline-block;\n",
       "  background: var(--xr-background-color);\n",
       "  padding-right: 10px;\n",
       "}\n",
       "\n",
       ".xr-attrs dd {\n",
       "  grid-column: 2;\n",
       "  white-space: pre-wrap;\n",
       "  word-break: break-all;\n",
       "}\n",
       "\n",
       ".xr-icon-database,\n",
       ".xr-icon-file-text2 {\n",
       "  display: inline-block;\n",
       "  vertical-align: middle;\n",
       "  width: 1em;\n",
       "  height: 1.5em !important;\n",
       "  stroke-width: 0;\n",
       "  stroke: currentColor;\n",
       "  fill: currentColor;\n",
       "}\n",
       "</style><pre class='xr-text-repr-fallback'>&lt;xarray.DataArray &#x27;tos&#x27; (j: 300, i: 360)&gt;\n",
       "array([[nan, nan, nan, ..., nan, nan, nan],\n",
       "       [nan, nan, nan, ..., nan, nan, nan],\n",
       "       [nan, nan, nan, ..., nan, nan, nan],\n",
       "       ...,\n",
       "       [nan, nan, nan, ..., nan, nan, nan],\n",
       "       [nan, nan, nan, ..., nan, nan, nan],\n",
       "       [nan, nan, nan, ..., nan, nan, nan]], dtype=float32)\n",
       "Coordinates:\n",
       "  * j          (j) int32 0 1 2 3 4 5 6 7 8 ... 292 293 294 295 296 297 298 299\n",
       "  * i          (i) int32 0 1 2 3 4 5 6 7 8 ... 352 353 354 355 356 357 358 359\n",
       "    latitude   (j, i) float64 -77.88 -77.88 -77.88 -77.88 ... 66.04 65.63 65.21\n",
       "    longitude  (j, i) float64 80.5 81.5 82.5 83.5 ... 79.94 79.96 79.97 79.99\n",
       "Attributes:\n",
       "    standard_name:  sea_surface_temperature\n",
       "    long_name:      Sea Surface Temperature\n",
       "    comment:        Temperature of upper boundary of the liquid ocean, includ...\n",
       "    units:          degC\n",
       "    cell_methods:   area: mean where sea time: mean\n",
       "    cell_measures:  area: areacello\n",
       "    history:        2019-11-08T18:23:54Z altered by CMOR: replaced missing va...\n",
       "    _ChunkSizes:    [  1 300 360]</pre><div class='xr-wrap' hidden><div class='xr-header'><div class='xr-obj-type'>xarray.DataArray</div><div class='xr-array-name'>'tos'</div><ul class='xr-dim-list'><li><span class='xr-has-index'>j</span>: 300</li><li><span class='xr-has-index'>i</span>: 360</li></ul></div><ul class='xr-sections'><li class='xr-section-item'><div class='xr-array-wrap'><input id='section-35c4e1cc-e7b9-477c-a070-71ca5be4b930' class='xr-array-in' type='checkbox' checked><label for='section-35c4e1cc-e7b9-477c-a070-71ca5be4b930' title='Show/hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-array-preview xr-preview'><span>nan nan nan nan nan nan nan nan ... nan nan nan nan nan nan nan nan</span></div><div class='xr-array-data'><pre>array([[nan, nan, nan, ..., nan, nan, nan],\n",
       "       [nan, nan, nan, ..., nan, nan, nan],\n",
       "       [nan, nan, nan, ..., nan, nan, nan],\n",
       "       ...,\n",
       "       [nan, nan, nan, ..., nan, nan, nan],\n",
       "       [nan, nan, nan, ..., nan, nan, nan],\n",
       "       [nan, nan, nan, ..., nan, nan, nan]], dtype=float32)</pre></div></div></li><li class='xr-section-item'><input id='section-40abc48a-0f82-43e8-97d4-9e56bfb17e8f' class='xr-section-summary-in' type='checkbox'  checked><label for='section-40abc48a-0f82-43e8-97d4-9e56bfb17e8f' class='xr-section-summary' >Coordinates: <span>(4)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>j</span></div><div class='xr-var-dims'>(j)</div><div class='xr-var-dtype'>int32</div><div class='xr-var-preview xr-preview'>0 1 2 3 4 5 ... 295 296 297 298 299</div><input id='attrs-9b75c77c-3056-43d1-8b2a-ed29be9faa17' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-9b75c77c-3056-43d1-8b2a-ed29be9faa17' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-a1b28cba-023e-4653-97f9-bcc8d08e77de' class='xr-var-data-in' type='checkbox'><label for='data-a1b28cba-023e-4653-97f9-bcc8d08e77de' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>units :</span></dt><dd>1</dd><dt><span>long_name :</span></dt><dd>cell index along second dimension</dd></dl></div><div class='xr-var-data'><pre>array([  0,   1,   2, ..., 297, 298, 299], dtype=int32)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>i</span></div><div class='xr-var-dims'>(i)</div><div class='xr-var-dtype'>int32</div><div class='xr-var-preview xr-preview'>0 1 2 3 4 5 ... 355 356 357 358 359</div><input id='attrs-3ad67619-6b3d-47c2-83be-3332d06b3e42' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-3ad67619-6b3d-47c2-83be-3332d06b3e42' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-8ed49374-00cc-4678-a938-580277770412' class='xr-var-data-in' type='checkbox'><label for='data-8ed49374-00cc-4678-a938-580277770412' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>units :</span></dt><dd>1</dd><dt><span>long_name :</span></dt><dd>cell index along first dimension</dd></dl></div><div class='xr-var-data'><pre>array([  0,   1,   2, ..., 357, 358, 359], dtype=int32)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>latitude</span></div><div class='xr-var-dims'>(j, i)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>-77.88 -77.88 ... 65.63 65.21</div><input id='attrs-e29932b5-c792-48e2-8dd6-d4f8a369a04c' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-e29932b5-c792-48e2-8dd6-d4f8a369a04c' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-bfde7c57-83d1-47be-9170-a66fc524613a' class='xr-var-data-in' type='checkbox'><label for='data-bfde7c57-83d1-47be-9170-a66fc524613a' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>standard_name :</span></dt><dd>latitude</dd><dt><span>long_name :</span></dt><dd>latitude</dd><dt><span>units :</span></dt><dd>degrees_north</dd><dt><span>bounds :</span></dt><dd>vertices_latitude</dd><dt><span>_ChunkSizes :</span></dt><dd>[300 360]</dd></dl></div><div class='xr-var-data'><pre>array([[-77.87662338, -77.87662338, -77.87662338, ..., -77.87662338,\n",
       "        -77.87662338, -77.87662338],\n",
       "       [-77.6297134 , -77.6297134 , -77.6297134 , ..., -77.6297134 ,\n",
       "        -77.6297134 , -77.6297134 ],\n",
       "       [-77.38170671, -77.38170671, -77.38170671, ..., -77.38170671,\n",
       "        -77.38170671, -77.38170671],\n",
       "       ...,\n",
       "       [ 65.20965413,  65.62409039,  66.03219689, ...,  66.03219689,\n",
       "         65.62409039,  65.20965413],\n",
       "       [ 65.21018119,  65.62564754,  66.0347538 , ...,  66.0347538 ,\n",
       "         65.62564754,  65.21018119],\n",
       "       [ 65.21044485,  65.62642645,  66.03603278, ...,  66.03603278,\n",
       "         65.62642645,  65.21044485]])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>longitude</span></div><div class='xr-var-dims'>(j, i)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>80.5 81.5 82.5 ... 79.97 79.99</div><input id='attrs-8e3a2081-d981-4f78-b59d-7bac59c8ee57' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-8e3a2081-d981-4f78-b59d-7bac59c8ee57' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-bac26c95-7066-442f-a79b-ddbb5ed950e3' class='xr-var-data-in' type='checkbox'><label for='data-bac26c95-7066-442f-a79b-ddbb5ed950e3' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>standard_name :</span></dt><dd>longitude</dd><dt><span>long_name :</span></dt><dd>longitude</dd><dt><span>units :</span></dt><dd>degrees_east</dd><dt><span>bounds :</span></dt><dd>vertices_longitude</dd><dt><span>_ChunkSizes :</span></dt><dd>[300 360]</dd></dl></div><div class='xr-var-data'><pre>array([[80.5       , 81.5       , 82.5       , ..., 77.5       ,\n",
       "        78.5       , 79.5       ],\n",
       "       [80.5       , 81.5       , 82.5       , ..., 77.5       ,\n",
       "        78.5       , 79.5       ],\n",
       "       [80.5       , 81.5       , 82.5       , ..., 77.5       ,\n",
       "        78.5       , 79.5       ],\n",
       "       ...,\n",
       "       [80.04428052, 80.13286833, 80.22153653, ..., 79.77846347,\n",
       "        79.86713167, 79.95571948],\n",
       "       [80.02659156, 80.07979084, 80.13303864, ..., 79.86696136,\n",
       "        79.92020916, 79.97340844],\n",
       "       [80.00887691, 80.02663615, 80.04441162, ..., 79.95558838,\n",
       "        79.97336385, 79.99112309]])</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-8c71a9c0-b9ba-4231-8e87-ee86c76d17ad' class='xr-section-summary-in' type='checkbox'  checked><label for='section-8c71a9c0-b9ba-4231-8e87-ee86c76d17ad' class='xr-section-summary' >Attributes: <span>(8)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><dl class='xr-attrs'><dt><span>standard_name :</span></dt><dd>sea_surface_temperature</dd><dt><span>long_name :</span></dt><dd>Sea Surface Temperature</dd><dt><span>comment :</span></dt><dd>Temperature of upper boundary of the liquid ocean, including temperatures below sea-ice and floating ice shelves.</dd><dt><span>units :</span></dt><dd>degC</dd><dt><span>cell_methods :</span></dt><dd>area: mean where sea time: mean</dd><dt><span>cell_measures :</span></dt><dd>area: areacello</dd><dt><span>history :</span></dt><dd>2019-11-08T18:23:54Z altered by CMOR: replaced missing value flag (-1e+20) with standard missing value (1e+20).</dd><dt><span>_ChunkSizes :</span></dt><dd>[  1 300 360]</dd></dl></div></li></ul></div></div>"
      ],
      "text/plain": [
       "<xarray.DataArray 'tos' (j: 300, i: 360)>\n",
       "array([[nan, nan, nan, ..., nan, nan, nan],\n",
       "       [nan, nan, nan, ..., nan, nan, nan],\n",
       "       [nan, nan, nan, ..., nan, nan, nan],\n",
       "       ...,\n",
       "       [nan, nan, nan, ..., nan, nan, nan],\n",
       "       [nan, nan, nan, ..., nan, nan, nan],\n",
       "       [nan, nan, nan, ..., nan, nan, nan]], dtype=float32)\n",
       "Coordinates:\n",
       "  * j          (j) int32 0 1 2 3 4 5 6 7 8 ... 292 293 294 295 296 297 298 299\n",
       "  * i          (i) int32 0 1 2 3 4 5 6 7 8 ... 352 353 354 355 356 357 358 359\n",
       "    latitude   (j, i) float64 -77.88 -77.88 -77.88 -77.88 ... 66.04 65.63 65.21\n",
       "    longitude  (j, i) float64 80.5 81.5 82.5 83.5 ... 79.94 79.96 79.97 79.99\n",
       "Attributes:\n",
       "    standard_name:  sea_surface_temperature\n",
       "    long_name:      Sea Surface Temperature\n",
       "    comment:        Temperature of upper boundary of the liquid ocean, includ...\n",
       "    units:          degC\n",
       "    cell_methods:   area: mean where sea time: mean\n",
       "    cell_measures:  area: areacello\n",
       "    history:        2019-11-08T18:23:54Z altered by CMOR: replaced missing va...\n",
       "    _ChunkSizes:    [  1 300 360]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "maximum"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "parliamentary-university",
   "metadata": {},
   "source": [
    "> **Direction matters**\n",
    ">\n",
    "> Calculating the daily maximum at each spatial (j, i) point requires taking information (i.e. the daily SST values) from each of the 321 chunks (because the the chunking was done along the time axis). Calculations like this that go across chunks are much slower than those long chunks. For instance, calculating the global maximum SST at each time step is very fast: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "unable-understanding",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 8.32 ms, sys: 1.01 ms, total: 9.34 ms\n",
      "Wall time: 10 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "spatial_max = dset['tos'].max(['j', 'i'], keep_attrs=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "communist-comedy",
   "metadata": {},
   "source": [
    "> It's worth being aware of the speed difference between along and across chunk calculations, but unfortunately we can't do much about it. The netCDF files are chunked by time, and changing the dask chunking to be inconsistent with the netCDF chunking (i.e. along a spatial axis) would make things even slower."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "accessible-remedy",
   "metadata": {},
   "source": [
    "TODO: Talk about writing your own dask aware functions?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "other-kennedy",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}