Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-enable SNAP but without Snappy #70

Merged
merged 6 commits into from
Apr 17, 2024
Merged

Re-enable SNAP but without Snappy #70

merged 6 commits into from
Apr 17, 2024

Conversation

griembauer
Copy link
Contributor

@griembauer griembauer commented Jan 23, 2024

In order to run i.sentinel.1pyrosargeocode as an alternative to snappy S-1 preprocessing, snappy has to be removed from the installation, but SNAP still needs to be installed with gpt enabled (see mundialis/esa-snap#37)


# GRASS GIS SETUP
COPY --from=grass /usr/local/bin/grass /usr/local/bin/grass
COPY --from=grass /usr/local/grass* /usr/local/grass/
# COPY --from=grass /usr/lib/gdalplugins/*_GRASS* /usr/lib/gdalplugins/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you don't copy the GDAL plugins, you might loose quite a lot of GDAL/OGR drivers, depending on the configuration of GDAL.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this commented line is just added in this PR, nothing was copied before either.

The missing GDAL plugins that are on the osgeo/grass-gis:releasebranch_8_3-alpine but not on the base image actinia:alpine-dependencies are:

- gdal_PNG.so
- gdal_GRASS.so
- ogr_GMLAS.so
- ogr_GRASS.so
- ogr_LIBKML.so
- ogr_MSSQLSpatial.so
- ogr_ODBC.so

but gdal_GRASS.so and ogr_GRASS.so are installed here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, so this Docker file uses GDAL from actinia:alpine-dependencies. In this case it would cause conflicts to copy GDAL plugins from osgeo/grass-gis:releasebranch_8_3-alpine. But GRASS has been compiled against GDAL in osgeo/grass-gis:releasebranch_8_3-alpine, therefore the GDAL versions in the two different images must be the same, otherwise you get errors running GRASS. Considering this, your solution to compile the GDAL GRASS plugin later on in this Dockerfile seems to be the safest option.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The GDAL versions in the grass image and actinia image are the same GDAL 3.6.4, released 2023/04/17 but this is definitely something we need to be aware of!

Comment on lines +139 to +145
# install GDAL GRASS plugin (required for i.sentinel1.pyrosargeocode)
RUN mkdir -p /usr/lib/gdalplugins
RUN git clone https://github.com/OSGeo/gdal-grass.git
RUN cd gdal-grass && ./configure --with-gdal=/usr/bin/gdal-config --with-grass=/usr/local/grass83 \
&& make && make install
RUN cd .. && rm -rf gdal-grass

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer these lines to be in the Dockerfile_alpine_dependencies. The goal is to have as much there with a scheduled build which can take as long as it needs and then to have as little things installed here as possible to make the build as quick as possible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, the GRASS installation comes from FROM osgeo/grass-gis:releasebranch_8_3-alpine as grass. This means that Dockerfile_alpine_dependencies does not have any GRASS installed, but the gdal-grass plugin needs to be installed with an existing GRASS and GDAL installation:

RUN cd gdal-grass && ./configure --with-gdal=/usr/bin/gdal-config --with-grass=/usr/local/grass83 \
&& make && make install

So moving the gdal-grass lines to Dockerfile_alpine_dependencies yields

0.816 configure: error: --with-grass=/usr/local/grass83 requested, but libraries not found!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right 😇
How long does it take to compile?
The multistage approach tries to have the best of everything - fast build times and most recent code. If it takes too long, we might need to come up with a different solution.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bit

RUN mkdir -p /usr/lib/gdalplugins
RUN git clone https://github.com/OSGeo/gdal-grass.git
RUN cd gdal-grass && ./configure --with-gdal=/usr/bin/gdal-config --with-grass=/usr/local/grass83 \
&& make && make install
RUN cd .. && rm -rf gdal-grass

takes ~ 10 seconds in the docker build command. What really takes time is the installation of GRASS addons (https://github.com/actinia-org/actinia-docker/blob/main/actinia-alpine/Dockerfile#L57) with 5-10 minutes in my local tests. All other lines only take a few seconds max

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok then fine for me!

Comment on lines 33 to 37
# commenting out snappy configuration:
# RUN (cd /root/.snap/snap-python/snappy && pip3 install .)
# RUN /usr/bin/python3 -c 'from snappy import ProductIO'
# RUN /usr/bin/python3 /root/.snap/about.py
# using gpt (graph processing tool) instead:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can do everything with gpt instead of snappy, no need to keep these comments here.

@griembauer griembauer marked this pull request as ready for review April 10, 2024 06:37
@griembauer
Copy link
Contributor Author

The referenced Alpine Dependencies image has been pushed to Dockerhub, the latest version has been successfully tested locally.

Copy link
Member

@mmacata mmacata left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks 🥳

@griembauer griembauer merged commit bd68ca5 into main Apr 17, 2024
@neteler neteler deleted the snap_no_snappy branch April 26, 2024 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants