Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Jupyter docker: new full build with latest of everything (#137)
# Overview New full build with latest of everything. ## Changes - New: save output of `conda env export` and environment size directly in the docker image for ease of tracking change between releases. - Updated pull request template for new docker release. - Jupyter env changes: - Unpin `libnetcdf` (Ouranosinc/PAVICS-landing#66 fixed). - Avoid `dask != 2024.11.0` due to bugs with Raven nb. - Relevant changes (alphabetical order): ```diff - - bokeh=3.4.1=pyhd8ed1ab_0 + - bokeh=3.5.2=pyhd8ed1ab_0 - - cartopy=0.23.0=py311h320fe9a_0 + - cartopy=0.24.0=py311h7db5c69_0 - - cf_xarray=0.9.0=pyhd8ed1ab_0 + - cf_xarray=0.10.0=pyhd8ed1ab_0 - - cfgrib=0.9.11.0=pyhd8ed1ab_0 + - cfgrib=0.9.14.1=pyhd8ed1ab_0 - - cftime=1.6.3=py311h1f0f07a_0 + - cftime=1.6.4=py311h9f3472d_1 - - climpred=2.4.0=pyhd8ed1ab_0 + - climpred=2.5.0=pyhd8ed1ab_0 - - clisops=0.13.0=pyhca7485f_0 + - clisops=0.14.1=pyhd8ed1ab_0 - - dask=2024.5.0=pyhd8ed1ab_0 + - dask=2024.10.0=pyhd8ed1ab_0 - - esmf=8.4.0=nompi_hdb2cfa9_4 + - esmf=8.6.1=nompi_h4441c20_3 - - fiona=1.9.1=py311h3f14cef_0 + - fiona=1.9.5=py311hf8e0aa6_2 - - gdal=3.6.2=py311hadb6153_6 + - gdal=3.8.5=py311hf92cf48_11 - - geopandas=0.14.4=pyhd8ed1ab_0 + - geopandas=1.0.1=pyhd8ed1ab_1 - - hvplot=0.9.2=pyhd8ed1ab_0 + - hvplot=0.11.1=pyhd8ed1ab_0 - - libnetcdf=4.8.1=nompi_h261ec11_106 + - libnetcdf=4.9.2=nompi_h135f659_114 - - numba=0.59.1=py311h96b013e_0 + - numba=0.60.0=py311h4bc866e_0 - - numpy=1.24.4=py311h64a7726_0 + - numpy=1.26.4=py311h64a7726_0 - - owslib=0.28.1=pyhd8ed1ab_0 + - owslib=0.32.0=pyhd8ed1ab_0 - - pandas=2.1.4=py311h320fe9a_0 + - pandas=2.2.3=py311h7db5c69_1 - - panel=1.4.2=pyhd8ed1ab_0 + - panel=1.5.3=pyhd8ed1ab_0 - - pydantic=2.7.1=pyhd8ed1ab_0 + - pydantic==2.7.4 - - pyogrio=0.5.1=py311h3f14cef_0 + - pyogrio=0.7.2=py311hf8e0aa6_1 - - python=3.11.6=hab00c5b_0_cpython + - python=3.11.10=hc5c86c4_3_cpython - - rasterio=1.3.6=py311h567e639_0 + - rasterio=1.3.10=py311h239598e_2 - - raven-hydro=0.2.4=py311h64a4d7b_0 + - raven-hydro=0.3.2=py311h81cb690_1 - - ravenpy==0.13.1 + - ravenpy=0.16.0=pyhd8ed1ab_0 - - rioxarray=0.15.5=pyhd8ed1ab_0 + - rioxarray=0.17.0=pyhd8ed1ab_0 - - roocs-utils=0.6.8=pyhd8ed1ab_0 + - roocs-utils=0.6.9=pyhd8ed1ab_0 - - scipy=1.13.0=py311h517d4fd_1 + - scipy=1.14.1=py311he9a78e4_1 - - shapely=2.0.1=py311h0f577a2_0 + - shapely=2.0.4=py311h0bed3d6_1 - - xarray=2023.8.0=pyhd8ed1ab_0 + - xarray=2024.9.0=pyhd8ed1ab_1 - - xclim=0.47.0=py311h38be061_0 + - xclim=0.53.2=pyhd8ed1ab_0 - - xesmf=0.8.5=pyhd8ed1ab_0 + - xesmf=0.8.8=pyhd8ed1ab_0 - - xscen=0.8.2=pyhd8ed1ab_0 + - xscen=0.10.1=pyhd8ed1ab_0 ``` ## Test - Deployed as "beta" image in production for bokeh visualization performance regression testing. - Manual test notebook https://github.com/Ouranosinc/PAVICS-landing/blob/master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb for bokeh visualization performance and it looks fine. - Jenkins build: - Default notebooks, all passed: https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/f8c1c585a4f1b17c0a3dc67453695deb2c30fb11/docker/saved_buildout/jenkins-buildlogs-default.txt - Raven notebooks, only known `HydroShare_integration.ipynb`: https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/0b3419bba4aa42ee5a1fa9a8de0c8a6b91bf8547/docker/saved_buildout/jenkins-buildlogs-raven.txt ## Related Issue / Discussion - Matching notebook fixes: - Pavics-sdi: Ouranosinc/pavics-sdi#336 - Finch: PR url - PAVICS-landing: Ouranosinc/PAVICS-landing#98 - RavenPy: CSHS-CWRA/RavenPy#395 - (...) - Deployment to PAVICS: bird-house/birdhouse-deploy#475 - Jenkins-config changes for new notebooks: PR url - Other issues found while working on this one - Issue 1 URL - Issue 2 URL - (...) - Previous release: #121 ## Additional Information Full diff conda env export: release-py311-240506-update240508...f8c1c58 Full new conda env export: https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/0b3419bba4aa42ee5a1fa9a8de0c8a6b91bf8547/docker/saved_buildout/conda-env-export.yml DockerHub build log https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/0b3419bba4aa42ee5a1fa9a8de0c8a6b91bf8547/docker/saved_buildout/docker-buildlogs.txt
- Loading branch information