Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add libgdal-jp2openjpeg dependency to ci_tests.yml #3328

Closed
wants to merge 1 commit into from

Conversation

weiji14
Copy link
Member

@weiji14 weiji14 commented Jul 15, 2024

Description of proposed changes

Enable reading *.jp2 files, such as the earth_relief dataset, and also mars_relief, earth_age, mercury_relief, earth_faa, moon_relief, earth_geoid, pluto_relief, earth_mag, venus_relief, earth_mag4km, earth_wdmam, earth_synbath, earth_vgg.

The gdal package on conda-forge now depends on libgdal-core which is a lightweight version that doesn't include certain drivers. See conda-forge/gdal-feedstock#948 and conda-forge/gdal-feedstock#722.

Fixes error on CI tests at https://github.com/GenericMappingTools/pygmt/actions/runs/9932182753/job/27433144596:

=================================== FAILURES ===================================
___________________ test_accessor_grid_source_file_not_exist ___________________
[gw0] linux -- Python 3.10.14 /home/runner/micromamba/envs/pygmt/bin/python3.10

    def test_accessor_grid_source_file_not_exist():
        """
        Check that the accessor fallbacks to the default registration and gtype when the
        grid source file (i.e., grid.encoding["source"]) doesn't exist.
        """
        # Load the 05m earth relief grid, which is stored as tiles
>       grid = load_earth_relief(
            resolution="05m", region=[0, 5, -5, 5], registration="pixel"
        )

../pygmt/tests/test_accessor.py:112: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../pygmt/datasets/earth_relief.py:175: in load_earth_relief
    grid = _load_remote_dataset(
../pygmt/helpers/decorators.py:773: in new_module
    return module_func(*bound.args, **bound.kwargs)
../pygmt/datasets/load_remote_dataset.py:418: in _load_remote_dataset
    grid = lib.virtualfile_to_raster(outgrid=None, vfname=voutgrd)
../pygmt/clib/session.py:2069: in virtualfile_to_raster
    return self.read_virtualfile(vfname, kind=kind).contents.to_dataarray()
../pygmt/datatypes/grid.py:192: in to_dataarray
    if grid[dim][0] > grid[dim][1]:
../../../../micromamba/envs/pygmt/lib/python3.10/site-packages/xarray/core/dataarray.py:772: in __getitem__
    return self.isel(indexers=self._item_key_to_dict(key))
../../../../micromamba/envs/pygmt/lib/python3.10/site-packages/xarray/core/dataarray.py:1291: in isel
    variable = self._variable.isel(indexers, missing_dims=missing_dims)
../../../../micromamba/envs/pygmt/lib/python3.10/site-packages/xarray/core/variable.py:1226: in isel
    return self[key]
../../../../micromamba/envs/pygmt/lib/python3.10/site-packages/xarray/core/variable.py:786: in __getitem__
    data = as_indexable(self._data)[indexer]
../../../../micromamba/envs/pygmt/lib/python3.10/site-packages/xarray/core/indexing.py:1481: in __getitem__
    result = self.array[key]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = Float64Index([], dtype='float64', name='y'), key = 0

    def __getitem__(self, key):
        """
        Override numpy.ndarray's __getitem__ method to work as desired.
    
        This function adds lists and Series as valid boolean indexers
        (ndarrays only supports ndarray with dtype=bool).
    
        If resulting ndim != 1, plain ndarray is returned instead of
        corresponding `Index` subclass.
    
        """
        getitem = self._data.__getitem__
    
        if is_integer(key) or is_float(key):
            # GH#44051 exclude bool, which would return a 2d ndarray
            key = com.cast_scalar_indexer(key, warn_float=True)
>           return getitem(key)
E           IndexError: index 0 is out of bounds for axis 0 with size 0

../../../../micromamba/envs/pygmt/lib/python3.10/site-packages/pandas/core/indexes/base.py:5320: IndexError
----------------------------- Captured stderr call -----------------------------
ERROR 4: `/home/runner/.gmt/server/earth/earth_relief/earth_relief_05m_p/S90E000.earth_relief_05m_p.jp2' not recognized as being in a supported file format. It could have been recognized by driver JP2OpenJPEG, but plugin gdal_JP2OpenJPEG.so is not available in your installation. You may install it with 'conda install -c conda-forge libgdal-jp2openjpeg'
ERROR 4: `/home/runner/.gmt/server/earth/earth_relief/earth_relief_05m_p/S90E000.earth_relief_05m_p.jp2' not recognized as being in a supported file format. It could have been recognized by driver JP2OpenJPEG, but plugin gdal_JP2OpenJPEG.so is not available in your installation. You may install it with 'conda install -c conda-forge libgdal-jp2openjpeg'
ERROR 4: `/home/runner/.gmt/server/earth/earth_relief/earth_relief_05m_p/S90E000.earth_relief_05m_p.jp2' not recognized as being in a supported file format. It could have been recognized by driver JP2OpenJPEG, but plugin gdal_JP2OpenJPEG.so is not available in your installation. You may install it with 'conda install -c conda-forge libgdal-jp2openjpeg'
[Session pygmt-session (103)]: Error returned from GMT API: GMT_FILE_NOT_FOUND (16)
[Session pygmt-session (103)]: Error returned from GMT API: GMT_GRID_READ_ERROR (18)
[Session pygmt-session (103)]: Error returned from GMT API: GMT_GRID_READ_ERROR (18)

Reminders

  • Run make format and make check to make sure the code follows the style guide.
  • Add tests for new features or tests that would have caught the bug that you're fixing.
  • Add new public functions/methods/classes to doc/api/index.rst.
  • Write detailed docstrings for all functions/methods.
  • If wrapping a new module, open a 'Wrap new GMT module' issue and submit reasonably-sized PRs.
  • If adding new functionality, add an example to docstrings or tutorials.
  • Use underscores (not hyphens) in names of Python files and directories.

Slash Commands

You can write slash commands (/command) in the first line of a comment to perform
specific operations. Supported slash command is:

  • /format: automatically format and lint the code

@weiji14 weiji14 added the maintenance Boring but important stuff for the core devs label Jul 15, 2024
@weiji14 weiji14 self-assigned this Jul 15, 2024
@@ -120,6 +120,7 @@ jobs:
python=${{ matrix.python-version }}${{ matrix.optional-packages }}
gmt=6.5.0
ghostscript=10.03.1
libgdal-jp2openjpeg
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to add this here, or in the gmt/pygmt conda-forge feedstock?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should add it to the gmt feedstock, otherwise gmt users also can't use these tiled remote datasets.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it'll be nicer to include it directly as a dependency in gmt-feedstock, I'll open a PR for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Boring but important stuff for the core devs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants