DSD installation guide

[veenstrajelmer]

Installation guide 2024

This installation guide prepares you for one of the breakout sessions of the Delft3D User Days - Day 1 about "Automated modelbuilding and postprocessing: Delft3D FM 1D2D and 2D3D models". In this session we will do some hands-on work with Python. The below installation guide provides you with a Python environment called dsd_env that contains the packages dfm_tools and hydromt_delft3dfm including all their dependencies. With this environment you can run all the example notebooks provided for this breakout session. You can also use the same environment for the breakout session "Deep-dive in pre- and postprocessing", since it also contains the packages HYDROLIB-core and xugrid. For those who were also present in 2023, the contents of the breakout sessions are very similar this year. Although, you can still attend if you want to know the latest updates (both robustness and functionality improvements) and you can ask any follow-up questions you may have.

Installation of Python environment

• Download Miniforge3 from the miniforge github and install it with the recommended settings.
• Open Miniforge Prompt
• conda create --name dsd_env python=3.11 git -c conda-forge -y (python 3.9 or 3.10 or 3.12 are also fine, git is required to download the course materials)
• conda activate dsd_env
• python -m pip install dfm_tools[examples] hydromt_delft3dfm[examples] (the latest versions are required, so 0.31.0 and 0.3.0)

Download course materials from github
Execute the following code block from the command line in a folder where you'd like to download the course materials to. Make sure you have activated the dsd_env environment before you do so:

mkdir dsd_delft3d_python_coursematerials
cd dsd_delft3d_python_coursematerials
curl https://codeload.github.com/deltares/hydromt_delft3dfm/tar.gz/main | tar -xz --strip=1 hydromt_delft3dfm-main/examples
ren examples hydromt_delft3dfm
curl https://codeload.github.com/deltares/dfm_tools/tar.gz/main | tar -xz --strip=2 dfm_tools-main/docs/notebooks
ren notebooks dfm_tools

You can also find all course materials online:

• hydromt_delft3dfm notebooks and example data: https://github.com/Deltares/hydromt_delft3dfm/tree/main/examples
• dfm_tools notebooks: https://github.com/Deltares/dfm_tools/blob/main/docs/notebooks

Register to retrieve CMEMS and ECMWF/CDS data (for dfm_tools modelbuilder)

• We will make use of publicly available data from Copernicus Programme of the European Union. To access this data you need to make two accounts at Copernicus Marine Service for CMEMS data and the ECMWF Climate Data Store for ERA5 data.
• Do not forget to accept the license agreements, for ECMWF also the one on this page
• You will be prompted for the CMEMS credentials and CDS/ECMWF apikey by the dfm_tools 2D3D modelbuilder notebook.

Doing the python exercises

• Open a command line in the folder where you downloaded the course materials: cd dsd_delft3d_python_coursematerials
• conda activate dsd_env
• Launch jupyter with: python -m jupyter notebook
• Open the notebook you want to work in and run it with SHIFT+ENTER or press the play button in the toolbar above

Installation of DeltaShell GUI (Delft3D FM Suite) (optional)

• If you want to run your newly generated model immediately, you require an FM executable. This is available after installing "Delft3D FM Suite 2024.03" (or a different version)
• Provided via download portal: https://download.deltares.nl/delft3d-user-days-break-out-session-modelbuilding-and-postprocessing (contains msi and license manager, login using MyDeltares account)
• You will receive a temporary license file via email

Installation of Open Earth Tools (OET) for MATLAB (optional)

• The Open Earth Tools contain an extensive set of MATLAB scripts and tools for pre- and post-processing of Delft3D model files. Note that a license is needed to use MATLAB.
• During the break-out session, we’ll focus on one of the available toolboxes for processing called EHY_tools.
• You can retrieve the OET (version controlled by SVN) by making a so-called check-out of the repository:
  • Download and install the latest version of TortoiseSVN from https://tortoisesvn.net/downloads.html
  • Make a check-out (using TortoiseSVN) of https://svn.oss.deltares.nl/repos/openearthtools/trunk/matlab
  • More information is available on: https://publicwiki.deltares.nl/display/OET/OpenEarth+tutorials#OpenEarthtutorials-GettingtheOpenEarthtools
• Add the OET to your MATLAB path:
  addpath('c:\your\path\to\check-out\OET\')
oetsettings

Additional info for organizers

• Presentations are available in: n:\Teams\DSD\DSD-INT 2024 - Delft Software Days\1.2 Delft3D User Days\Break-out sessions\1. Modelbuilding and postprocessing
• In 2024 the msi/GUI is provided via the Deltares download portal (see link above), the course materials can be downloaded directly from Github (see above). Alternatively, download the latest course materials before the course and add them to course materials zipfile on the download portal.
• Edward copy the contents of this issue in an email to the participants, including a license file as attachment.
• Prepare slide+pitch to promote the breakout session during the DSD morning program
• Reconsider planning in overview_slides.pptx, there is again 1.5+1=2.5 hours available
• ERA5 download takes ages (also with new CDS, at 27-9-2024). Check this again in a few weeks and maybe ask participants to comment ERA5.
• Streamline docs for DSD hydromt_delft3dfm#198
• Make sure all participants have wifi (guest) access before the breakout session starts
• Check the dsd_env environment:
  • run dfm_tools example notebooks
  • test/run model created by dfm_tools modelbuilder notebook >> runs in interacter and via batfile, but crashes when running in Delft3D FM Suite 2024.03 HMWQ (https://issuetracker.deltares.nl/browse/D3DFMIQ-3802). Commenting ExtForceFile in the mdu is a workaround to make the model run, but no output will be shown afterwards.
  • run hydromt_delft3dfm example notebooks
  • test/run models created by different hydromt_delft3dfm notebooks >> in interacter and in Delft3D FM Suite 2024.03 1D2D without errors, not sure if the models are generated as expected