MimtPy

MIami Multi Track tools in Python
MimtpyApp works based on the S1***.he5, exclude_data.txt and inputs floder generated by Mintpy.

1. Installation:

Test only on Linux, not sure about MacOS and Windows

1.1 Download and setup

Run the following in your terminal to download the development version of MimiPy

cd rsmas_insar/source/  
git clone https://github.com/geodesymiami/mimtpy.git  

Seting the following environment variables in your environment.bash

export MIMTPY_HOME=${RSMASINSAR_HOME}/sources/MimtPy  
export PYTHONPATH=${PYTHONPATH}:${MIMTPY_HOME}  
export PATH=${PATH}:${MIMTPY_HOME}/mimtpy  

1.2. the packages used in MimtPy that should be installed

via conda

rasterio
geopandas
osgeo

install Kite following 6.3

Kite

2. Directory structure

2.1 Required structure (from MintPy):

$SCRATCHDIR/GalapagosSenDT128
├── mintpy
    ├── S1_IW1_128_0593_0597_20141213_20180607.he5

Some applications (which?) require the original interferogram stack and geometry file

$SCRATCHDIR/GalapagosSenDT128
├── mintpy
    ├── S1_IW1_128_0593_0597_20141213_20180607.he5
    ├── inputs
        ├── geometryRadar.h5
        ├── ifgramStack.he5

2.2 Folder structure after runing mimtpyApp.py:

$SCRATCHDIR/GalapagosSenDT128
├── mintpy
    ├── S1_IW1_128_0593_0597_20141213_20180607.he5
    ├── inputs
    |   ├── geometryRadar.h5
    |   ├── ifgramStack.he5
    ├── velocity
    |   ├── velocity_date1_date2.h5 
    ├── displacement
        ├── displacement_date1_date2.h5 

3. Runing MimtPy

3.1 Routine workflow of mimtpyApp.py

MimtPy only use HDFEOS file and geometryRadar.h5 file generated by MintPy. In order to run mimtpyApp.py, you must organize the folder structure as 2.the Pre-requirement of MimtPy.

mimtpyApp.py -h/--help #help
mimtpyApp.py <template> # run mimtpy using mimtpyApp.template

The template example can be found in samples folder mimtpyApp.template

Inside the mimtpyApp.py it can generate displacement or velocity between two dates; do concatenation of adjacent trakcs; calculate horizontal and vertical data and plot figures. What's more, mimtpyApp.py can process one or more project as the same time, according to user's setting.

3.2 Functions

Singletrack scripts to prepare for modelling

• save_kite.py: generate Kite scene using HDFEOS data.

• save_insamp.py: prepare data for Barnhart's InSAMP software (written in Matlab). Please note that the data used by this script must be geocoded.

• save_geodmod.py: generate files used by Geodmod software.

• convert_Kite2GBIS.py: convert Kite scene to GBIS input data.

Other singletrack scripts

• HDFEOS_to_geotiff.py: transferring S1**.he5 file into *.h5 and *.geotiff files based on given dataset name.

• multi_transects.py: Extract data along the profiles which are perpendicular to the fault

• H5UNW_to_geotiff.py: transferring *.unw or *.h5 file into geotiff file.

• generate_track_polygon.py: Get the footprint scene of each project

Multitrack scripts

• mimtpyApp.py: generate files according to mimtpyApp.template

• track_offsets.py: concatenate adjacent tracks

• overlap2horz_vert.py: Calculate horizontal and vertical value from the overlap region between adjacent tracks

Plot scripts

• plot_geotiff.py: plot geotiff files with given *.shp files and gps velocity vector field.

Modelling scripts:

• generate_script_RELAX_V2.py: generate template files for RELAX. People must given one example file.

• gridsearch_ramps_batch_relax: doing gridseach batchlly

• gridsearch_ramps_relax: doing gridsearch for one pair of parameter

Other utilities:

• synthetic_S1.py: simulate linear ramps

4. Access to mintpy data products

Browse through all directories using: https://129.114.104.223/data/HDF5EOS

Get them via wget using:

wget -r -nH --cut-dirs=2 --no-parent --reject="index.html*" --no-check-certificate https://129.114.104.223/data/HDF5EOS/unittestGalapagosSenDT128/mintpy

If you have access to jetstream and your public key is added in the .ssh directory:

scp -rp centos@129.114.104.223:/data/HDF5EOS/KashgarSenAT129 .

5. links to software packages

for Kite: git clone https://github.com/pyrocko/kite.git

for InSAMP: git clone https://github.com/williamBarnhart/InSamp.git

6. Installation of modelling software

We are using the RELAX and BEAT packages. Here some installation notes:

6.1 RELAX

It is very easy. First downloading the RELAX package. Then runing the following command to use it.

source setup.sh

6.2 Bayesian Earthquake Analysis Tool (BEAT)

You first need to install Pyrocko. We tried the conda installation of Pyrocko and BEAT. It worked for pyrocko but we ran into a problem with BEAT. We got an error message regarding the numpy installation as shown below, which we need to resolve. The pip installation works, but is more complex.

For the conda installation we followed the manual of Pyrocko. and BEAT.

Here a quick summary of what I tried:

conda install -c pyrocko pyrocko
conda create -n beat python=3.6
conda activate beat
source activate
conda deactivate
conda activate beat
cd $RSMAS_INSAR/sources
git clone https://github.com/hvasbath/beat
conda install -n beat libgfortran openblas theano pygpu openmpi pandas numpy openmpi
cat > ~/.theanorc   as described in manual (adjust -L flag to your conda BEAT environment)
python3 test/numpy_test.py

But then the testing gave an error:

python3 test/numpy_test.py
Traceback (most recent call last):
  File "/scratch/05861/tg851601/code1/rsmas_insar/3rdparty/miniconda3/lib/python3.8/site-packages/numpy/core/__init__.py", line 22, in <module>
    from . import multiarray
  File "/scratch/05861/tg851601/code1/rsmas_insar/3rdparty/miniconda3/lib/python3.8/site-packages/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/scratch/05861/tg851601/code1/rsmas_insar/3rdparty/miniconda3/lib/python3.8/site-packages/numpy/core/overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test/numpy_test.py", line 1, in <module>
    import numpy as np
  File "/scratch/05861/tg851601/code1/rsmas_insar/3rdparty/miniconda3/lib/python3.8/site-packages/numpy/__init__.py", line 145, in <module>
    from . import core
  File "/scratch/05861/tg851601/code1/rsmas_insar/3rdparty/miniconda3/lib/python3.8/site-packages/numpy/core/__init__.py", line 48, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

• The Python version is: Python3.6 from "/scratch/05861/tg851601/code1/rsmas_insar/3rdparty/miniconda3/envs/beat/bin/python3"
• The NumPy version is: "1.20.1"

and make sure that they are the versions you expect. Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy.core._multiarray_umath'

Please note that after installation BEAT, you need to install fomosto-qseis、fomosto-qssp and fomosto-psgrn-pscmp to calculate Green Functions. Using fomosto-qseis as an example, the installation steps are:

git clone https://git.pyrocko.org/pyrocko/fomoto-qseis.git fomosto-qseis
cd fomosto-qseis
autoreconf -i  
./configure
make
sudo make install

6.3 Kite

You first need to install Pyrocko. Then use conda and pip to install Kite. The installation commonds are:

conda install pyqtgraph
pip install git+https://github.com/Turbo87/utm
pip install git+https://github.com/pyrocko/kite