Add libgdal-jp2openjpeg dependency #291

weiji14 · 2024-07-15T03:00:51Z

Enable reading tiled .jp2 files from https://docs.generic-mapping-tools.org/latest/datasets/remote-data.html such as earth_relief, mars_relief, earth_age, mercury_relief, earth_faa, moon_relief, earth_geoid, pluto_relief, earth_mag, venus_relief, earth_mag4km, earth_wdmam, earth_synbath, earth_vgg.

The gdal package on conda-forge now depends on libgdal-core which is a lightweight version that doesn't include certain drivers. See conda-forge/gdal-feedstock#948 and conda-forge/gdal-feedstock#722.

Checklist

Used a personal fork of the feedstock to propose changes
Bumped the build number (if the version is unchanged)
Reset the build number to 0 (if the version changed)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
Ensured the license file is being packaged.

Closes GenericMappingTools/pygmt#3328

To test for existence of libgdal-jp2openjpeg driver.

conda-forge-webservices · 2024-07-15T03:00:57Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

weiji14 · 2024-07-15T03:14:35Z

recipe/meta.yaml

+    # Test reading/writing a JP2 file (S90E000.earth_relief_05m_p.jp2)
+    - gmt grdcut @earth_relief_05m -R-10/-9/3/5 -G/tmp/reliefcut.jp2=gd:JP2OpenJPEG


Hmm, this doesn't error with a non-zero exit code at https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=978661&view=logs&j=656edd35-690f-5c53-9ba3-09c10d0bea97&t=986b1512-c876-5f92-0d81-ba851554a0a3&l=4457:

+ gmt grdcut @earth_relief_05m -R-10/-9/3/5 -G/tmp/reliefcut.jp2=gd:JP2OpenJPEG grdblend [NOTICE]: Remote data courtesy of GMT data server oceania [http://oceania.generic-mapping-tools.org/] grdblend [NOTICE]: SRTM15 Earth Relief v2.6 at 05x05 arc minutes reduced by Gaussian Cartesian filtering (26.2 km fullwidth) [Tozer et al., 2019]. grdblend [NOTICE]: -> Download 180x180 degree grid tile (earth_relief_05m_g): S90W180 ERROR 4: `/home/conda/.gmt/server/earth/earth_relief/earth_relief_05m_g/S90W180.earth_relief_05m_g.jp2' not recognized as being in a supported file format. It could have been recognized by driver JP2OpenJPEG, but plugin gdal_JP2OpenJPEG.so is not available in your installation. You may install it with 'conda install -c conda-forge libgdal-jp2openjpeg' grdblend [ERROR]: ERROR - Unable to convert SRTM file /home/conda/.gmt/server/earth/earth_relief/earth_relief_05m_g/S90W180.earth_relief_05m_g.jp2 to compressed netCDF format grdblend [ERROR]: Unable to obtain remote file @S90W180.earth_relief_05m_g.nc ERROR 4: `/home/conda/.gmt/server/earth/earth_relief/earth_relief_05m_g/S90W180.earth_relief_05m_g.jp2' not recognized as being in a supported file format. It could have been recognized by driver JP2OpenJPEG, but plugin gdal_JP2OpenJPEG.so is not available in your installation. You may install it with 'conda install -c conda-forge libgdal-jp2openjpeg' grdblend [ERROR]: ERROR - Unable to convert SRTM file /home/conda/.gmt/server/earth/earth_relief/earth_relief_05m_g/S90W180.earth_relief_05m_g.jp2 to compressed netCDF format grdblend [ERROR]: Cannot find file @S90W180.earth_relief_05m_g.nc ERROR 4: `/home/conda/.gmt/server/earth/earth_relief/earth_relief_05m_g/S90W180.earth_relief_05m_g.jp2' not recognized as being in a supported file format. It could have been recognized by driver JP2OpenJPEG, but plugin gdal_JP2OpenJPEG.so is not available in your installation. You may install it with 'conda install -c conda-forge libgdal-jp2openjpeg' grdblend [ERROR]: ERROR - Unable to convert SRTM file /home/conda/.gmt/server/earth/earth_relief/earth_relief_05m_g/S90W180.earth_relief_05m_g.jp2 to compressed netCDF format grdblend [ERROR]: Cannot find file @S90W180.earth_relief_05m_g.nc grdblend [ERROR]: File @S90W180.earth_relief_05m_g.nc not found [Session gmt (0)]: Error returned from GMT API: GMT_FILE_NOT_FOUND (16) grdcut [ERROR]: ERROR - Unable to produce blended grid from /tmp/=tiled_333_GX_YV6NSp [Session gmt (0)]: Error returned from GMT API: GMT_GRID_READ_ERROR (18) [Session gmt (0)]: Error returned from GMT API: GMT_GRID_READ_ERROR (18) [Session gmt (0)]: Error returned from GMT API: GMT_GRID_READ_ERROR (18) + exit 0

@seisman, is there a better to do this JP2 driver check?

Maybe just run gdal-config --formats | grep JP2OpenJPEG?

That won't error with a non-zero exit code (and JP2OpenJPEG seems to be listed even though libgdal-jp2openjpeg isn't installed).

gmt grdcut @earth_relief_05m -R-10/-9/3/5 -G/tmp/reliefcut.jp2=gd:JP2OpenJPEG

Maybe check if /tmp/reliefcut.jp2 exists?

Ok, this seems to work:

gmt grdcut @earth_relief_05m -R-10/-9/3/5 -G/tmp/reliefcut.jp2=gd:JP2OpenJPEG [ -e '/tmp/reliefcut.jp2' ] && exit 0 || exit 1

errors as intended at https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=978819&view=logs&j=656edd35-690f-5c53-9ba3-09c10d0bea97&t=986b1512-c876-5f92-0d81-ba851554a0a3&l=4480:

+ gmt grdcut @earth_relief_05m -R-10/-9/3/5 -G/tmp/reliefcut.jp2=gd:JP2OpenJPEG grdblend [NOTICE]: Remote data courtesy of GMT data server oceania [http://oceania.generic-mapping-tools.org/] grdblend [NOTICE]: SRTM15 Earth Relief v2.6 at 05x05 arc minutes reduced by Gaussian Cartesian filtering (26.2 km fullwidth) [Tozer et al., 2019]. grdblend [NOTICE]: -> Download 180x180 degree grid tile (earth_relief_05m_g): S90W180 ERROR 4: `/home/conda/.gmt/server/earth/earth_relief/earth_relief_05m_g/S90W180.earth_relief_05m_g.jp2' not recognized as being in a supported file format. It could have been recognized by driver JP2OpenJPEG, but plugin gdal_JP2OpenJPEG.so is not available in your installation. You may install it with 'conda install -c conda-forge libgdal-jp2openjpeg' grdblend [ERROR]: ERROR - Unable to convert SRTM file /home/conda/.gmt/server/earth/earth_relief/earth_relief_05m_g/S90W180.earth_relief_05m_g.jp2 to compressed netCDF format grdblend [ERROR]: Unable to obtain remote file @S90W180.earth_relief_05m_g.nc ERROR 4: `/home/conda/.gmt/server/earth/earth_relief/earth_relief_05m_g/S90W180.earth_relief_05m_g.jp2' not recognized as being in a supported file format. It could have been recognized by driver JP2OpenJPEG, but plugin gdal_JP2OpenJPEG.so is not available in your installation. You may install it with 'conda install -c conda-forge libgdal-jp2openjpeg' grdblend [ERROR]: ERROR - Unable to convert SRTM file /home/conda/.gmt/server/earth/earth_relief/earth_relief_05m_g/S90W180.earth_relief_05m_g.jp2 to compressed netCDF format grdblend [ERROR]: Cannot find file @S90W180.earth_relief_05m_g.nc ERROR 4: `/home/conda/.gmt/server/earth/earth_relief/earth_relief_05m_g/S90W180.earth_relief_05m_g.jp2' not recognized as being in a supported file format. It could have been recognized by driver JP2OpenJPEG, but plugin gdal_JP2OpenJPEG.so is not available in your installation. You may install it with 'conda install -c conda-forge libgdal-jp2openjpeg' grdblend [ERROR]: ERROR - Unable to convert SRTM file /home/conda/.gmt/server/earth/earth_relief/earth_relief_05m_g/S90W180.earth_relief_05m_g.jp2 to compressed netCDF format grdblend [ERROR]: Cannot find file @S90W180.earth_relief_05m_g.nc grdblend [ERROR]: File @S90W180.earth_relief_05m_g.nc not found [Session gmt (0)]: Error returned from GMT API: GMT_FILE_NOT_FOUND (16) grdcut [ERROR]: ERROR - Unable to produce blended grid from /tmp/=tiled_333_GX_TH7dZO [Session gmt (0)]: Error returned from GMT API: GMT_GRID_READ_ERROR (18) [Session gmt (0)]: Error returned from GMT API: GMT_GRID_READ_ERROR (18) [Session gmt (0)]: Error returned from GMT API: GMT_GRID_READ_ERROR (18) + '[' -e /tmp/reliefcut.jp2 ']' + exit 1

Gonna wait to see if this works on Windows though 🙂 Edit: yeah, the test fails on Windows as expected, but on the grdcut command rather than the [ -e '/tmp/reliefcut.jp2'] command...

the test fails on Windows as expected, but on the grdcut command rather than the [ -e '/tmp/reliefcut.jp2'] command...

Maybe the /tmp directory doesn't exist on Windows?

Oh yeah, maybe just write to the current directory. Feel free to push to this branch, it's late for me already so won't get to it until tomorrow.

Xref https://unix.stackexchange.com/questions/474244/one-liner-to-check-for-file-exists

conda-forge-webservices · 2024-07-15T08:48:15Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe) and found some lint.

Here's what I've got...

For recipe:

Failed to even lint the recipe, probably because of a conda-smithy bug 😢. This likely indicates a problem in your meta.yaml, though. To get a traceback to help figure out what's going on, install conda-smithy and run conda smithy recipe-lint . from the recipe directory.

This reverts commit 3825db4.

Square brackets might be mistaken for selectors.

conda-forge-webservices · 2024-07-15T08:51:31Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

This reverts commit d5e48d1.

weiji14 · 2024-07-15T09:19:19Z

@conda-forge-admin, please rerender

github-actions · 2024-07-15T09:20:56Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/gmt-feedstock/actions/runs/9936852688.

recipe/meta.yaml

seisman · 2024-07-15T12:39:40Z

Tests pass and the PR is merged. Please also apply the same fix to the dev branch when you have time.

Cherry-picked from 058b79f/conda-forge#291

weiji14 · 2024-07-16T04:30:58Z

Tests pass and the PR is merged. Please also apply the same fix to the dev branch when you have time.

Included as part of #292

* Bump GMT dev version from 6.6.0.dev0+eec0fc5 to 6.6.0.dev1+708d76b * Fix incorrect edits from yq Need to update bumpdevversion.yml script later. * Add libgdal-jp2openjpeg as runtime dependency Cherry-picked from 058b79f/#291 * Actually add libgdal-jp2openjpeg as a runtime dependency --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Add unit test for reading/writing a JP2 file

e2c3a43

To test for existence of libgdal-jp2openjpeg driver.

Add libgdal-jp2openjpeg as runtime dependency

3825db4

weiji14 commented Jul 15, 2024

View reviewed changes

Return exit code 1 if /tmp/reliefcut.jp2 file doesn't exist

67f2a78

Xref https://unix.stackexchange.com/questions/474244/one-liner-to-check-for-file-exists

weiji14 added 2 commits July 15, 2024 20:48

Revert "Add libgdal-jp2openjpeg as runtime dependency"

d5e48d1

This reverts commit 3825db4.

Put command around double quotes

12204d3

Square brackets might be mistaken for selectors.

Reapply "Add libgdal-jp2openjpeg as runtime dependency"

c06550d

This reverts commit d5e48d1.

seisman reviewed Jul 15, 2024

View reviewed changes

recipe/meta.yaml Outdated Show resolved Hide resolved

recipe/meta.yaml Outdated Show resolved Hide resolved

Save the result to the current directory

12c83a0

seisman marked this pull request as ready for review July 15, 2024 12:38

seisman requested review from leouieda and ocefpaf as code owners July 15, 2024 12:38

seisman merged commit 058b79f into conda-forge:main Jul 15, 2024
7 checks passed

weiji14 deleted the libgdal-jp2openjpeg branch July 15, 2024 19:34

weiji14 added a commit to GenericMappingTools/gmt-feedstock that referenced this pull request Jul 16, 2024

Add libgdal-jp2openjpeg as runtime dependency

1c30dfd

Cherry-picked from 058b79f/conda-forge#291

weiji14 mentioned this pull request Jul 20, 2024

Wrap GMT's standard data type GMT_IMAGE for images GenericMappingTools/pygmt#3338

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add libgdal-jp2openjpeg dependency #291

Add libgdal-jp2openjpeg dependency #291

weiji14 commented Jul 15, 2024 •

edited

Loading

conda-forge-webservices bot commented Jul 15, 2024

weiji14 Jul 15, 2024

seisman Jul 15, 2024

weiji14 Jul 15, 2024 •

edited

Loading

seisman Jul 15, 2024

weiji14 Jul 15, 2024 •

edited

Loading

seisman Jul 15, 2024

weiji14 Jul 15, 2024

conda-forge-webservices bot commented Jul 15, 2024

conda-forge-webservices bot commented Jul 15, 2024

weiji14 commented Jul 15, 2024

github-actions bot commented Jul 15, 2024

seisman commented Jul 15, 2024

weiji14 commented Jul 16, 2024

		# Test reading/writing a JP2 file (S90E000.earth_relief_05m_p.jp2)
		- gmt grdcut @earth_relief_05m -R-10/-9/3/5 -G/tmp/reliefcut.jp2=gd:JP2OpenJPEG

Add libgdal-jp2openjpeg dependency #291

Add libgdal-jp2openjpeg dependency #291

Conversation

weiji14 commented Jul 15, 2024 • edited Loading

conda-forge-webservices bot commented Jul 15, 2024

weiji14 Jul 15, 2024

Choose a reason for hiding this comment

seisman Jul 15, 2024

Choose a reason for hiding this comment

weiji14 Jul 15, 2024 • edited Loading

Choose a reason for hiding this comment

seisman Jul 15, 2024

Choose a reason for hiding this comment

weiji14 Jul 15, 2024 • edited Loading

Choose a reason for hiding this comment

seisman Jul 15, 2024

Choose a reason for hiding this comment

weiji14 Jul 15, 2024

Choose a reason for hiding this comment

conda-forge-webservices bot commented Jul 15, 2024

conda-forge-webservices bot commented Jul 15, 2024

weiji14 commented Jul 15, 2024

github-actions bot commented Jul 15, 2024

seisman commented Jul 15, 2024

weiji14 commented Jul 16, 2024

weiji14 commented Jul 15, 2024 •

edited

Loading

weiji14 Jul 15, 2024 •

edited

Loading

weiji14 Jul 15, 2024 •

edited

Loading