-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
update nbs, tutorial data and html versions #78
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
``` _ PAVICS-landing-fix_nbs_jupyter_alpha_refresh_output/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb::Cell 0 _ Notebook cell execution failed Cell 0: Cell outputs differ Input: import warnings import xarray as xr from IPython.display import display # Fancy representation of xarray objects from siphon.catalog import TDSCatalog warnings.simplefilter("ignore") url = "https://pavics.ouranos.ca/twitcher/ows/proxy/thredds/catalog/datasets/simulations/bias_adjusted/cmip5/ouranos/cb-oura-1.0/catalog.xml" # TEST_USE_PROD_DATA # Create Catalog cat = TDSCatalog(url) # DAP link for this demo ds_url = cat.datasets[0].access_urls["OPENDAP"] # xarray.Dataset ds = xr.open_dataset(ds_url, chunks=dict(time=256 * 2, lon=32, lat=32)) display(ds) a = ds.tasmin.isel(time=0).plot(figsize=(10, 4)) Traceback: mismatch 'text/plain' assert reference_output == test_output failed: '<xarray.Data...id: NCC' == '<xarray.Data...id: NCC' Skipping 957 identical leading characters in diff, use -v to show - driving_experiment: historical,rcp85 + driving_model: NorESM1-M ... ... frequency: day modeling_realm: atmos target_dataset: CANADA : ANUSPLIN interpolated Canada daily 3... target_dataset_references: CANADA : https://doi.org/10.1175/2011BAMS3132... driving_institution: Norwegian Climate Centre driving_institute_id: NCC _ PAVICS-landing-fix_nbs_jupyter_alpha_refresh_output/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb::Cell 1 _ Notebook cell execution failed Cell 1: Cell outputs differ Input: from clisops.core import subset lon = [-75.4, -85, -65.5] # Longitude lat = [46.67, 41, 55.3] # Latitude ds_gridpoint = subset.subset_gridpoint(ds, lon=lon, lat=lat) display(ds_gridpoint) # Plot first year of tasmax data a = ds_gridpoint.tasmax.isel(time=slice(0, 365)).plot.line(x="time", figsize=(10, 4)) Traceback: mismatch 'text/plain' assert reference_output == test_output failed: '<xarray.Data...id: NCC' == '<xarray.Data...id: NCC' Skipping 892 identical leading characters in diff, use -v to show - driving_experiment: historical,rcp85 + driving_model: NorESM1-M ... ... frequency: day modeling_realm: atmos target_dataset: CANADA : ANUSPLIN interpolated Canada daily 3... target_dataset_references: CANADA : https://doi.org/10.1175/2011BAMS3132... driving_institution: Norwegian Climate Centre driving_institute_id: NCC _ PAVICS-landing-fix_nbs_jupyter_alpha_refresh_output/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb::Cell 3 _ Notebook cell execution failed Cell 3: Cell outputs differ Input: # Specify the longitude and latitude boundaries lon_bnds = [-80.5, -60.2] lat_bnds = [44, 55] ds1 = subset.subset_bbox(ds, lon_bnds=lon_bnds, lat_bnds=lat_bnds) display(ds1) # Plot a map of first timestep a = ds1.tasmax.isel(time=0).plot() Traceback: mismatch 'text/plain' assert reference_output == test_output failed: '<xarray.Data...id: NCC' == '<xarray.Data...id: NCC' Skipping 955 identical leading characters in diff, use -v to show - driving_experiment: historical,rcp85 + driving_model: NorESM1-M ... ... frequency: day modeling_realm: atmos target_dataset: CANADA : ANUSPLIN interpolated Canada daily 3... target_dataset_references: CANADA : https://doi.org/10.1175/2011BAMS3132... driving_institution: Norwegian Climate Centre driving_institute_id: NCC ```
``` _ PAVICS-landing-fix_nbs_jupyter_alpha_refresh_output/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb::Cell 2 _ Notebook cell execution failed Cell 2: Cell outputs differ Input: import xclim # Access some station data url = "https://pavics.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/datasets/station_obs/ECCC_AHCCD_gen3_temperature.ncml" ds_stat = xr.open_dataset(url, chunks=dict(station=1)) istat = 115 # Compare use of differernt percent tolerance levels (tolerance == 0.1) plt.figure(figsize=(15, 10)) color = ["blue", "green", "orange"] for i, tol in enumerate([0.25, 0.1, 0.05]): plt.subplot(3, 1, i + 1) with xclim.set_options( check_missing="pct", missing_options={"pct": {"tolerance": tol}} ): # compute yearly max tasmax tx_mean = xclim.atmos.tx_mean( tasmax=ds_stat.isel(station=istat).tasmax, freq="YS" ) tx_mean.plot( marker="o", color=color[i], label=f"{len(tx_mean.dropna('time').values)} \ valid years w/ <={tol*100}% missing data", ) plt.title("") plt.xlabel("") if i == 0: name = str(ds_stat.isel(station=istat).station_name.values) plt.title(f"{name.capitalize().replace('_',' ')} Station") elif i == 2: plt.xlabel("time") plt.legend(loc="lower right") Traceback: Unexpected output fields from running code: {'stderr'} ```
Attempt to fix the following error: ``` _ PAVICS-landing-fix_nbs_jupyter_alpha_refresh_output/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb::Cell 2 _ Notebook cell execution failed Cell 2: Cell outputs differ Input: import xclim # Access some station data url = "https://pavics.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/datasets/station_obs/ECCC_AHCCD_gen3_temperature.ncml" ds_stat = xr.open_dataset(url, chunks=dict(station=1)) istat = 115 # Compare use of differernt percent tolerance levels (tolerance == 0.1) plt.figure(figsize=(15, 10)) color = ["blue", "green", "orange"] for i, tol in enumerate([0.25, 0.1, 0.05]): plt.subplot(3, 1, i + 1) with xclim.set_options( check_missing="pct", missing_options={"pct": {"tolerance": tol}} ): # compute yearly max tasmax tx_mean = xclim.atmos.tx_mean( tasmax=ds_stat.isel(station=istat).tasmax, freq="YS" ) tx_mean.plot( marker="o", color=color[i], label=f"{len(tx_mean.dropna('time').values)} \ valid years w/ <={tol*100}% missing data", ) plt.title("") plt.xlabel("") if i == 0: name = str(ds_stat.isel(station=istat).station_name.values) plt.title(f"{name.capitalize().replace('_',' ')} Station") elif i == 2: plt.xlabel("time") plt.legend(loc="lower right") Traceback: Missing output fields from running code: {'stderr'} ```
…nbs_jupyter_alpha_refresh_output
…into fix_nbs_jupyter_alpha � Conflicts: � content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb � content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb � content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.html � content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb � content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb � content/notebooks/climate_indicators/output.zip � src/assets/notebooks/PAVICStutorial_ClimateDataAnalysis-4Ensembles.html
for more information, see https://pre-commit.ci
tlvu
added a commit
to Ouranosinc/PAVICS-e2e-workflow-tests
that referenced
this pull request
May 9, 2024
…t xclim and ravenpy to smooth transition (#121) # Overview This new full build has latest of almost everything except `xclim` and `ravenpy` as intermediate step to smooth transition to `pandas` 2.2 freq strings changes. ## Changes - New: save conda env export, DockerHub build logs and Jenkins test result in the repo to track changes much more easily between releases - Jenkins: add `SAVE_RESULTING_NOTEBOOK_TIMEOUT` for slow notebooks or slow machine - Jupyter env changes: - add `conda-pack` so we can export the conda env outside of the docker image if need to run locally without docker - upgrade from Python 3.9 to 3.11 - Relevant changes (alphabetical order): ```diff - - birdy=0.8.4=pyh1a96a4e_0 + - birdhouse-birdy==0.8.7 # major upgrade from v2 to v3 - - bokeh=2.4.3=pyhd8ed1ab_3 + - bokeh=3.4.1=pyhd8ed1ab_0 - - cartopy=0.21.1=py39h6e7ad6e_0 + - cartopy=0.23.0=py311h320fe9a_0 - - cf_xarray=0.8.0=pyhd8ed1ab_0 + - cf_xarray=0.9.0=pyhd8ed1ab_0 - - cfgrib=0.9.10.4=pyhd8ed1ab_0 + - cfgrib=0.9.11.0=pyhd8ed1ab_0 - - cftime=1.6.2=py39h2ae25f5_1 + - cftime=1.6.3=py311h1f0f07a_0 - - climpred=2.3.0=pyhd8ed1ab_0 + - climpred=2.4.0=pyhd8ed1ab_0 - - clisops=0.9.6=pyh1a96a4e_0 + - clisops=0.13.0=pyhca7485f_0 - - dask=2023.5.1=pyhd8ed1ab_0 + - dask=2024.5.0=pyhd8ed1ab_0 - - geopandas=0.13.0=pyhd8ed1ab_0 + - geopandas=0.14.4=pyhd8ed1ab_0 - - hvplot=0.8.3=pyhd8ed1ab_0 + - hvplot=0.9.2=pyhd8ed1ab_0 - - numpy=1.23.5=py39h3d75532_0 + - numpy=1.24.4=py311h64a7726_0 - - numba=0.57.0=py39hb75a051_1 + - numba=0.59.1=py311h96b013e_0 # major upgrade from v1 to v2 - - pandas=1.3.5=py39hde0f152_0 + - pandas=2.1.4=py311h320fe9a_0 # major upgrade to v1 - - panel=0.14.4=pyhd8ed1ab_0 + - panel=1.4.2=pyhd8ed1ab_0 # major upgrade from v1 to v2 - - pydantic=1.10.8=py39hd1e30aa_0 + - pydantic=2.7.1=pyhd8ed1ab_0 # Python 3.9 to 3.11 - - python=3.9.16=h2782a2a_0_cpython + - python=3.11.6=hab00c5b_0_cpython - - raven-hydro=0.2.1=py39h8e2dbb5_1 + - raven-hydro=0.2.4=py311h64a4d7b_0 - - ravenpy=0.12.1=py39hf3d152e_0 + - ravenpy==0.13.1 - - rioxarray=0.14.1=pyhd8ed1ab_0 + - rioxarray=0.15.5=pyhd8ed1ab_0 - - roocs-utils=0.6.4=pyh1a96a4e_0 + - roocs-utils=0.6.8=pyhd8ed1ab_0 - - scipy=1.9.1=py39h8ba3f38_0 + - scipy=1.13.0=py311h517d4fd_1 - - xarray=2023.1.0=pyhd8ed1ab_0 + - xarray=2023.8.0=pyhd8ed1ab_0 - - xclim=0.43.0=py39hf3d152e_1 + - xclim=0.47.0=py311h38be061_0 - - xesmf=0.7.1=pyhd8ed1ab_0 + - xesmf=0.8.5=pyhd8ed1ab_0 - - xskillscore=0.0.24=pyhd8ed1ab_0 + - xskillscore=0.0.26=pyhd8ed1ab_0 + - xscen=0.8.2=pyhd8ed1ab_0 + - figanos==0.3.0 - - xncml==0.2 + - xncml==0.4.0 ``` ## Test - Deployed as "beta" image in production for bokeh visualization performance regression testing. - Manual test notebook https://github.com/Ouranosinc/PAVICS-landing/blob/master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb for bokeh visualization performance and it looks fine. - Jenkins build: - Default notebooks, all passed: https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/54792e6510adfcd1bb21e1bd31fdfa36c5c634e0/docker/saved_buildout/jenkins-buildlogs-default.txt - Raven notebooks, only known `HydroShare_integration.ipynb` failing: https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/931cfc924a147d07b59e88badff9f170e852a03b/docker/saved_buildout/jenkins-buildlogs-raven.txt ## Related Issue / Discussion - Matching notebook fixes: - Pavics-sdi: PR Ouranosinc/pavics-sdi#321 - Finch: PR url: None - PAVICS-landing: PR Ouranosinc/PAVICS-landing#78 - RavenPy: PR CSHS-CWRA/RavenPy#356 - Resolves Ouranosinc/PAVICS-landing#65 - Resolves Ouranosinc/PAVICS-landing#66 - Deployment to PAVICS: bird-house/birdhouse-deploy#453 - Jenkins-config changes for new notebooks: PR url: None - Other issues found while working on this one - computationalmodelling/nbval#204 - jupyterlab-contrib/jupyter-archive#132 - CSHS-CWRA/RavenPy#357 - CSHS-CWRA/RavenPy#361 - CSHS-CWRA/RavenPy#362 - Previous release: PR #134 ## Additional Information Full diff conda env export: 81deb99...931cfc9#diff-e8f2a6a53085ae29bb7cedc701c1d345a330651ae971555e85a5c005e94f4cd9 Full new conda env export: https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/931cfc924a147d07b59e88badff9f170e852a03b/docker/saved_buildout/conda-env-export.yml DockerHub build log https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/931cfc924a147d07b59e88badff9f170e852a03b/docker/saved_buildout/docker-buildlogs.txt
tlvu
approved these changes
May 9, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fix issues related to https://github.com/Ouranosinc/pavics-jupyter-env-issues/issues/2, https://github.com/Ouranosinc/pavics-jupyter-env-issues/issues/6
Fix for Ouranosinc/PAVICS-e2e-workflow-tests#121
Replace #68