Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

grdimage shading fails for xarray input #364

Closed
liamtoney opened this issue Nov 2, 2019 · 18 comments · Fixed by #649
Closed

grdimage shading fails for xarray input #364

liamtoney opened this issue Nov 2, 2019 · 18 comments · Fixed by #649
Labels
bug Something isn't working scipy-sprint Things to work on during the SciPy sprint! upstream Bug or missing feature of upstream core GMT
Milestone

Comments

@liamtoney
Copy link
Member

Description of the problem

Passing shading=True in a simple fig.grdimage() call produces an error if the input grid is an xarray.DataArray. The error does not appear when I use the special "@" filename.

Full code that generated the error

import pygmt

fig = pygmt.Figure()

dem = pygmt.datasets.load_earth_relief('10m')

fig.grdimage(dem, region=[-180, 180, -90, 90], frame='af',
             projection='Cyl_stere/6i', cmap='geo', shading=True)

fig.show(method='external')

This works, though:

import pygmt

fig = pygmt.Figure()

fig.grdimage('@earth_relief_10m', region=[-180, 180, -90, 90], frame='af',
             projection='Cyl_stere/6i', cmap='geo', shading=True)

fig.show(method='external')

Error message summary

grdimage [ERROR]: Passing max <= zmin prevents automatic CPT generation!
grdimage [ERROR]: Failed to read CPT geo.
Full error message (debug verbosity)
grdimage [DEBUG]: Map distance calculation will be using great circle approximation with authalic auxiliary latitudes and authalic (R_2) radius = 6371007.1809 m, in meter.
grdimage [DEBUG]: Look for file -180/180/-90/90 in /Users/liamtoney/.gmt
grdimage [DEBUG]: Look for file -180/180/-90/90 in /Users/liamtoney/.gmt/cache
grdimage [DEBUG]: Look for file -180/180/-90/90 in /Users/liamtoney/.gmt/server
grdimage [DEBUG]: Look for file -180/180/-90/90 in /Users/liamtoney/.gmt/server/srtm1
grdimage [DEBUG]: Look for file -180/180/-90/90 in /Users/liamtoney/.gmt/server/srtm3
grdimage [DEBUG]: Got regular w/e/s/n for region (-180/180/-90/90)
grdimage [DEBUG]: Look for file geo in /Users/liamtoney/.gmt
grdimage [DEBUG]: Look for file geo in /Users/liamtoney/.gmt/cache
grdimage [DEBUG]: Look for file geo in /Users/liamtoney/.gmt/server
grdimage [DEBUG]: Look for file geo in /Users/liamtoney/.gmt/server/srtm1
grdimage [DEBUG]: Look for file geo in /Users/liamtoney/.gmt/server/srtm3
grdimage [DEBUG]: Look for file @GMTAPI@-000000 in /Users/liamtoney/.gmt
grdimage [DEBUG]: Look for file @GMTAPI@-000000 in /Users/liamtoney/.gmt/cache
grdimage [DEBUG]: Look for file @GMTAPI@-000000 in /Users/liamtoney/.gmt/server
grdimage [DEBUG]: Look for file @GMTAPI@-000000 in /Users/liamtoney/.gmt/server/srtm1
grdimage [DEBUG]: Look for file @GMTAPI@-000000 in /Users/liamtoney/.gmt/server/srtm3
grdimage [INFORMATION]: Read header from file @GMTAPI@-000000
grdimage [DEBUG]: Local file /Users/liamtoney/.gmt/server/gmt_hash_server.txt found
grdimage [DEBUG]: File /Users/liamtoney/.gmt/server/gmt_hash_server.txt less than 24 hours old, refresh is premature.
grdimage [DEBUG]: api_begin_io: Input resource access is now enabled [container]
grdimage [DEBUG]: api_import_grid: Passed ID = 0 and mode = 1
grdimage [DEBUG]: Grid/Image dimensions imply w/e/s/n = -180/180/-90/90, inc = 0.166667/0.166667, gridline registration, n_layers = 1
grdimage [DEBUG]: Geographic input grid, longitudes span exactly 360
grdimage [INFORMATION]: Given domain implies x_inc = 0.166667
grdimage [INFORMATION]: Given domain implies y_inc = 0.166667
grdimage [DEBUG]: Grid is considered to have a 360-degree longitude range.
grdimage [DEBUG]: Chosen boundary condition for all edges: geographic
grdimage [DEBUG]: Geographic input grid, longitudes span exactly 360
grdimage [DEBUG]: Object ID 1 : Registered Grid Memory Reference 7fc1ced29590 as an Input resource with geometry Surface [n_objects = 2]
grdimage [DEBUG]: Successfully created a new Grid container
grdimage [DEBUG]: GMT_End_IO: Input resource access is now disabled
grdimage [WARNING]: Spherical approximation used!
grdimage [WARNING]: Central meridian not given, default to 0
grdimage [DEBUG]: Projected values in meters: -2.00151e+07 2.00151e+07 -1.2742e+07 1.2742e+07
grdimage [DEBUG]: Auto-frame interval for axis 0 item 0: d = 60  f = 15
grdimage [INFORMATION]: Auto-frame interval for x-axis (item 0): a60f15
grdimage [DEBUG]: Auto-frame interval for axis 1 item 0: d = 60  f = 15
grdimage [INFORMATION]: Auto-frame interval for y-axis (item 0): a60f15
grdimage [INFORMATION]: Map scale is 2626.65 km per cm or 1:2.62665e+08.
grdimage [DEBUG]: Projected grid is non-orthogonal, nonlinear, or dpi was changed
grdimage [INFORMATION]: Derive intensity grid from data grid
grdimage [DEBUG]: Object ID 2 : Registered Grid Memory Copy 7fc1cec5a210 as an Output resource with geometry Surface [n_objects = 3]
grdimage [DEBUG]: Successfully created a new Grid container
grdimage [INFORMATION]: Calling grdgradient with args -G@GMTAPI@-000002 -A-45.0 -Nt1 -R-180/180/-90/90 --GMT_HISTORY=false @GMTAPI@-000000
grdimage [DEBUG]: GMT now running in modern mode [Session ID = 9323]
grdimage (gmtlib_free_tmp_arrays): tried to free unallocated memory
grdgradient [DEBUG]: History: Process -R-180/180/-90/90
grdgradient [DEBUG]: Look for file -180/180/-90/90 in /Users/liamtoney/.gmt
grdgradient [DEBUG]: Look for file -180/180/-90/90 in /Users/liamtoney/.gmt/cache
grdgradient [DEBUG]: Look for file -180/180/-90/90 in /Users/liamtoney/.gmt/server
grdgradient [DEBUG]: Look for file -180/180/-90/90 in /Users/liamtoney/.gmt/server/srtm1
grdgradient [DEBUG]: Look for file -180/180/-90/90 in /Users/liamtoney/.gmt/server/srtm3
grdgradient [DEBUG]: Got regular w/e/s/n for region (-180/180/-90/90)
grdgradient [DEBUG]: Look for file -45.0 in /Users/liamtoney/.gmt
grdgradient [DEBUG]: Look for file -45.0 in /Users/liamtoney/.gmt/cache
grdgradient [DEBUG]: Look for file -45.0 in /Users/liamtoney/.gmt/server
grdgradient [DEBUG]: Look for file -45.0 in /Users/liamtoney/.gmt/server/srtm1
grdgradient [DEBUG]: Look for file -45.0 in /Users/liamtoney/.gmt/server/srtm3
grdgradient [DEBUG]: amplitude_set = Y offset_set = N sigma_set = N
grdgradient [INFORMATION]: Processing input grid
grdgradient [DEBUG]: api_begin_io: Input resource access is now enabled [container]
grdgradient [DEBUG]: api_import_grid: Passed ID = 0 and mode = 1
grdgradient [DEBUG]: Grid/Image dimensions imply w/e/s/n = -180/180/-90/90, inc = 0.166667/0.166667, gridline registration, n_layers = 1
grdgradient [DEBUG]: Geographic input grid, longitudes span exactly 360
grdgradient [INFORMATION]: Given domain implies x_inc = 0.166667
grdgradient [INFORMATION]: Given domain implies y_inc = 0.166667
grdgradient [DEBUG]: Grid is considered to have a 360-degree longitude range.
grdgradient [DEBUG]: Chosen boundary condition for all edges: geographic
grdgradient [DEBUG]: Geographic input grid, longitudes span exactly 360
grdgradient [DEBUG]: Object ID 3 : Registered Grid Memory Reference 7fc1cc5eb9f0 as an Input resource with geometry Surface [n_objects = 4]
grdgradient [DEBUG]: Successfully created a new Grid container
grdgradient [DEBUG]: GMT_End_IO: Input resource access is now disabled
grdgradient [DEBUG]: api_begin_io: Input resource access is now enabled [container]
grdgradient [DEBUG]: api_import_grid: Passed ID = 0 and mode = 2
grdgradient [INFORMATION]: Importing grid data from user memory location
grdgradient [DEBUG]: Grid is considered to have a 360-degree longitude range.
grdgradient [DEBUG]: Chosen boundary condition for all edges: geographic
grdgradient [INFORMATION]: Set boundary condition for top    edge: geographic
grdgradient [INFORMATION]: Set boundary condition for bottom edge: geographic
grdgradient [DEBUG]: GMT_End_IO: Input resource access is now disabled
grdgradient [DEBUG]: Object ID 4 : Registered Grid Memory Reference 7fc1cc5d3850 as an Input resource with geometry Surface [n_objects = 5]
grdgradient [DEBUG]: Successfully duplicated a Grid
grdgradient [INFORMATION]:  Min Mean Max sigma intensities:
grdgradient [INFORMATION]: 	-0.35723270421	0.000249671630837	0.283594327547	0.0132078509689
grdgradient [DEBUG]: GMT_Write_Data: Writing Grid to memory object 2 from object 4 which transfers ownership
grdgradient [DEBUG]: api_begin_io: Output resource access is now enabled [container]
grdgradient [DEBUG]: api_export_data: Messenger dummy output container for object 2 [item 2] freed and set resource=data=NULL
grdgradient [DEBUG]: api_export_grid: Passed ID = 2 and mode = 0
grdgradient [INFORMATION]: Referencing grid data to GMT_GRID memory location
grdgradient [DEBUG]: GMT_End_IO: Output resource access is now disabled
grdgradient [DEBUG]: gmtapi_garbage_collection: Destroying object: C=0 A=0 ID=3 W=Input F=Grid M=Memory Copy S=Used P=7fc1cc5eb9f0 N=(null)
grdgradient [DEBUG]: GMTAPI_Garbage_Collection freed 1 memory objects
grdgradient [DEBUG]: gmtapi_unregister_io: Unregistering object no 3 [n_objects = 4]
grdgradient [DEBUG]: gmtapi_unregister_io: Unregistering object no 4 [n_objects = 3]
grdgradient (gmtlib_free_tmp_arrays): tried to free unallocated memory
grdimage [DEBUG]: Running in PS mode modern
grdimage [DEBUG]: Use PS filename /Users/liamtoney/.gmt/sessions/gmt6.9323/gmt_8.ps-
grdimage [DEBUG]: Create hidden PS file /Users/liamtoney/.gmt/sessions/gmt6.9323/gmt_8.ps-
grdimage [DEBUG]: Got session name as pygmt-session and default graphics formats as pdf
grdimage [INFORMATION]: Allocate and read data from file @GMTAPI@-000000
grdimage [DEBUG]: api_begin_io: Input resource access is now enabled [container]
grdimage [DEBUG]: api_import_grid: Passed ID = 1 and mode = 2
grdimage [INFORMATION]: Duplicating grid data from GMT_GRID memory location
grdimage [DEBUG]: GMT_End_IO: Input resource access is now disabled
grdimage [INFORMATION]: Allocates memory and read intensity file
grdimage [INFORMATION]: project grid files
grdimage [DEBUG]: Object ID 5 : Registered Grid Memory Reference 7fc1cc765be0 as an Input resource with geometry Surface [n_objects = 4]
grdimage [DEBUG]: Successfully duplicated a Grid
grdimage [DEBUG]: gmt_project_init: IN: Inc [0/0] n_columns/n_rows [2161/1081] dpi = 0 offset = 0
grdimage [DEBUG]: gmt_project_init: OUT: Inc [0/0] n_columns/n_rows [2161/1081] dpi = 0 offset = 0
grdimage [INFORMATION]: Grid projection from size 2161x1081 to 2161x1081
grdimage [DEBUG]: Successfully added data array to previously registered Grid container
grdimage [DEBUG]: gmt_grd_project: In [-180/180/-90/90] and out [0/6/0/3.81971863421]
grdimage [DEBUG]: GMT_Destroy_Data: freed memory for a Grid for object 1
grdimage [DEBUG]: gmtapi_unregister_io: Unregistering object no 1 [n_objects = 3]
grdimage [DEBUG]: gmtapi_unregister_io: Object no 1 has non-NULL resource pointer
grdimage [DEBUG]: Object ID 6 : Registered Grid Memory Reference 7fc1cc73a3e0 as an Input resource with geometry Surface [n_objects = 4]
grdimage [DEBUG]: Successfully duplicated a Grid
grdimage [DEBUG]: gmt_project_init: IN: Inc [0/0] n_columns/n_rows [2161/1081] dpi = 0 offset = 0
grdimage [DEBUG]: gmt_project_init: OUT: Inc [0/0] n_columns/n_rows [2161/1081] dpi = 0 offset = 0
grdimage [INFORMATION]: Grid projection from size 2161x1081 to 2161x1081
grdimage [DEBUG]: Successfully added data array to previously registered Grid container
grdimage [DEBUG]: gmt_grd_project: In [-180/180/-90/90] and out [0/6/0/3.81971863421]
grdimage [WARNING]: gmt_grd_project: Output grid extrema [-1.89669/1.87961] exceed extrema of input grid [-0.97649/0.970346] due to resampling
grdimage [WARNING]: gmt_grd_project: Output grid clipped to input grid extrema
grdimage [DEBUG]: GMT_Destroy_Data: freed memory for a Grid for object 2
grdimage [DEBUG]: gmtapi_unregister_io: Unregistering object no 2 [n_objects = 3]
grdimage [DEBUG]: gmtapi_unregister_io: Object no 2 has non-NULL resource pointer
grdimage [INFORMATION]: Given grid header, select default CPT to be turbo
grdimage [DEBUG]: Look for file geo in /Users/liamtoney/.gmt
grdimage [DEBUG]: Look for file geo in /Users/liamtoney/.gmt/cache
grdimage [DEBUG]: Look for file geo in /Users/liamtoney/.gmt/server
grdimage [DEBUG]: Look for file geo in /Users/liamtoney/.gmt/server/srtm1
grdimage [DEBUG]: Look for file geo in /Users/liamtoney/.gmt/server/srtm3
grdimage [ERROR]: Passing max <= zmin prevents automatic CPT generation!
grdimage [ERROR]: Failed to read CPT geo.
grdimage [DEBUG]: gmtapi_garbage_collection: Destroying object: C=0 A=0 ID=5 W=Input F=Grid M=Memory Reference S=Unused P=7fc1cc765be0 N=(null)
grdimage [DEBUG]: gmtapi_garbage_collection: Destroying object: C=0 A=0 ID=6 W=Input F=Grid M=Memory Reference S=Unused P=7fc1cc73a3e0 N=(null)
grdimage [DEBUG]: GMTAPI_Garbage_Collection freed 2 memory objects
grdimage [DEBUG]: gmtapi_unregister_io: Unregistering object no 5 [n_objects = 2]
grdimage [DEBUG]: gmtapi_unregister_io: Unregistering object no 6 [n_objects = 1]
grdimage (gmtlib_free_tmp_arrays): tried to free unallocated memory
Error: /syntaxerror in --%ztokenexec_continue--
Operand stack:
   PSL_draw_path_lines
Execution stack:
   %interp_exit   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push   --nostringval--   --nostringval--   --nostringval--   false   1   %stopped_push   1999   1   3   %oparray_pop   1998   1   3   %oparray_pop   1982   1   3   %oparray_pop   1868   1   3   %oparray_pop   --nostringval--   %errorexec_pop   .runexec2   --nostringval--   --nostringval--   --nostringval--   2   %stopped_push   --nostringval--
Dictionary stack:
   --dict:981/1684(ro)(G)--   --dict:0/20(G)--   --dict:78/200(L)--   --dict:136/250(L)--
Current allocation mode is local
Last OS error: No such file or directory
psconvert [ERROR]: System call [gs -q -dSAFER -dNOPAUSE -dBATCH -sDEVICE=bbox -DPSL_no_pagefill -dMaxBitmap=2147483647 -dUseFastColor=true '/Users/liamtoney/.gmt/sessions/gmt6.9323/gmt_8.ps-' 2> '/Users/liamtoney/.gmt/sessions/gmt6.9323/psconvert_9323c.bb'] returned error 256.
---------------------------------------------------------------------------
GMTCLibError                              Traceback (most recent call last)
<ipython-input-21-0836d05457e7> in <module>
      8              projection='Cyl_stere/6i', cmap='geo', shading=True, V='d')
      9
---> 10 fig.show(method='external')

~/repos/pygmt/pygmt/figure.py in show(self, dpi, width, method)
    265             raise GMTInvalidInput("Invalid show method '{}'.".format(method))
    266         if method == "external":
--> 267             pdf = self._preview(fmt="pdf", dpi=dpi, anti_alias=False, as_bytes=False)
    268             launch_external_viewer(pdf)
    269             img = None

~/repos/pygmt/pygmt/figure.py in _preview(self, fmt, dpi, as_bytes, **kwargs)
    340         """
    341         fname = os.path.join(self._preview_dir.name, "{}.{}".format(self._name, fmt))
--> 342         self.savefig(fname, dpi=dpi, **kwargs)
    343         if as_bytes:
    344             with open(fname, "rb") as image:

~/repos/pygmt/pygmt/figure.py in savefig(self, fname, transparent, crop, anti_alias, show, **kwargs)
    223             kwargs["W"] = "+k"
    224
--> 225         self.psconvert(prefix=prefix, fmt=fmt, crop=crop, **kwargs)
    226         if show:
    227             launch_external_viewer(fname)

~/repos/pygmt/pygmt/helpers/decorators.py in new_module(*args, **kwargs)
    181                 if alias in kwargs:
    182                     kwargs[arg] = kwargs.pop(alias)
--> 183             return module_func(*args, **kwargs)
    184
    185         new_module.aliases = aliases

~/repos/pygmt/pygmt/helpers/decorators.py in new_module(*args, **kwargs)
    289                         )
    290             # Execute the original function and return its output
--> 291             return module_func(*args, **kwargs)
    292
    293         return new_module

~/repos/pygmt/pygmt/figure.py in psconvert(self, **kwargs)
    163             kwargs["A"] = ""
    164         with Session() as lib:
--> 165             lib.call_module("psconvert", build_arg_string(kwargs))
    166
    167     def savefig(

~/repos/pygmt/pygmt/clib/session.py in call_module(self, module, args)
    487             raise GMTCLibError(
    488                 "Module '{}' failed with status code {}:\n{}".format(
--> 489                     module, status, self._error_message
    490                 )
    491             )

GMTCLibError: Module 'psconvert' failed with status code 78:
psconvert [ERROR]: System call [gs -q -dSAFER -dNOPAUSE -dBATCH -sDEVICE=bbox -DPSL_no_pagefill -dMaxBitmap=2147483647 -dUseFastColor=true '/Users/liamtoney/.gmt/sessions/gmt6.9323/gmt_8.ps-' 2> '/Users/liamtoney/.gmt/sessions/gmt6.9323/psconvert_9323c.bb'] returned error 256.

System information

  • Operating system: macOS Catalina 10.15.1
  • Python installation (Anaconda, system, ETS): miniconda
  • Version of GMT: 6.0.0
  • Version of Python: 3.7.3
  • Version of this package: 55af9b6
  • If using conda, paste the output of conda list below:
output of conda list
# packages in environment at /Users/liamtoney/miniconda3/envs/pygmt:
#
# Name                    Version                   Build  Channel
alabaster                 0.7.12                     py_0    conda-forge
appdirs                   1.4.3                      py_1    conda-forge
appnope                   0.1.0                 py37_1000    conda-forge
asn1crypto                1.2.0                    py37_0    conda-forge
astroid                   2.3.2                    py37_0    conda-forge
atomicwrites              1.3.0                      py_0    conda-forge
attrs                     19.3.0                     py_0    conda-forge
babel                     2.7.0                      py_0    conda-forge
backcall                  0.1.0                      py_0    conda-forge
black                     19.3b0                     py_0    conda-forge
bleach                    3.1.0                      py_0    conda-forge
boost-cpp                 1.70.0               h75728bb_2    conda-forge
bzip2                     1.0.8                h01d97ff_1    conda-forge
ca-certificates           2019.9.11            hecc5488_0    conda-forge
cairo                     1.16.0            he1c11cd_1002    conda-forge
certifi                   2019.9.11                py37_0    conda-forge
cffi                      1.13.1           py37h33e799b_0    conda-forge
cfitsio                   3.470                h84d2f63_2    conda-forge
cftime                    1.0.4.2          py37h3b54f70_0    conda-forge
chardet                   3.0.4                 py37_1003    conda-forge
click                     7.0                        py_0    conda-forge
coverage                  4.5.4            py37h01d97ff_0    conda-forge
cryptography              2.7              py37h212c5bf_0    conda-forge
curl                      7.65.3               h22ea746_0    conda-forge
cycler                    0.10.0                     py_2    conda-forge
dbus                      1.13.6               h2f22bb5_0    conda-forge
dcw-gmt                   1.1.4                      1001    conda-forge
decorator                 4.4.0                      py_0    conda-forge
defusedxml                0.6.0                      py_0    conda-forge
docutils                  0.15.2                   py37_0    conda-forge
entrypoints               0.3                   py37_1000    conda-forge
expat                     2.2.5             h4a8c4bd_1004    conda-forge
ffmpeg                    4.2                  h5c2b479_0    conda-forge
fftw                      3.3.8           nompi_h9629793_1109    conda-forge
flake8                    3.7.8                    py37_1    conda-forge
fontconfig                2.13.1            h6b1039f_1001    conda-forge
freetype                  2.10.0               h24853df_1    conda-forge
freexl                    1.0.5             h1de35cc_1002    conda-forge
fribidi                   1.0.5             h01d97ff_1002    conda-forge
gdal                      3.0.1           py37h97c3584_10    conda-forge
geos                      3.7.2                h6de7cb9_2    conda-forge
geotiff                   1.5.1                h7e8e167_4    conda-forge
gettext                   0.19.8.1          h46ab8bc_1002    conda-forge
ghostscript               9.22              h0a44026_1001    conda-forge
giflib                    5.1.7                h01d97ff_1    conda-forge
glib                      2.58.3            h9d45998_1002    conda-forge
gmp                       6.1.2             h0a44026_1000    conda-forge
gmt                       6.0.0                h5e3c8ef_0    conda-forge
gnuplot                   5.2.7                h14e7342_3    conda-forge
gnutls                    3.6.5             h53004b3_1002    conda-forge
graphicsmagick            1.3.33               h04fbb9f_0    conda-forge
graphite2                 1.3.13            h2098e52_1000    conda-forge
gshhg-gmt                 2.3.7                      1002    conda-forge
harfbuzz                  2.4.0                hd8d2a14_3    conda-forge
hdf4                      4.2.13            hf3c6af0_1002    conda-forge
hdf5                      1.10.5          nompi_h3e39495_1104    conda-forge
icu                       64.2                 h6de7cb9_1    conda-forge
idna                      2.8                   py37_1000    conda-forge
imagesize                 1.1.0                      py_0    conda-forge
importlib_metadata        0.23                     py37_0    conda-forge
ipykernel                 5.1.3            py37h5ca1d4c_0    conda-forge
ipython                   7.8.0            py37h5ca1d4c_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                7.5.1                      py_0    conda-forge
isort                     4.3.21                   py37_0    conda-forge
jedi                      0.15.1                   py37_0    conda-forge
jinja2                    2.10.3                     py_0    conda-forge
jpeg                      9c                h1de35cc_1001    conda-forge
json-c                    0.13.1            h1de35cc_1001    conda-forge
jsonschema                3.1.1                    py37_0    conda-forge
jupyter                   1.0.0                      py_2    conda-forge
jupyter_client            5.3.3                    py37_1    conda-forge
jupyter_console           6.0.0                      py_0    conda-forge
jupyter_core              4.5.0                      py_0    conda-forge
kealib                    1.4.10            h6659575_1005    conda-forge
kiwisolver                1.1.0            py37h770b8ee_0    conda-forge
krb5                      1.16.3            hcfa6398_1001    conda-forge
lame                      3.100             h1de35cc_1001    conda-forge
lazy-object-proxy         1.4.2            py37h01d97ff_0    conda-forge
libblas                   3.8.0               14_openblas    conda-forge
libcblas                  3.8.0               14_openblas    conda-forge
libclang                  8.0.1                h770b8ee_1    conda-forge
libcurl                   7.65.3               h16faf7d_0    conda-forge
libcxx                    9.0.0                h89e68fa_1    conda-forge
libdap4                   3.20.4               habf5908_0    conda-forge
libedit                   3.1.20170329      hcfe32e1_1001    conda-forge
libffi                    3.2.1             h6de7cb9_1006    conda-forge
libgd                     2.2.5             h5f83dcc_1006    conda-forge
libgdal                   3.0.1               h0158245_10    conda-forge
libgfortran               4.0.0                         2    conda-forge
libiconv                  1.15              h01d97ff_1005    conda-forge
libkml                    1.3.0             hed7d534_1010    conda-forge
liblapack                 3.8.0               14_openblas    conda-forge
libllvm8                  8.0.1                h770b8ee_0    conda-forge
libnetcdf                 4.7.1           nompi_he14bcb6_101    conda-forge
libopenblas               0.3.7                h4bb4525_2    conda-forge
libpng                    1.6.37               h2573ce8_0    conda-forge
libpq                     11.5                 h756f0eb_1    conda-forge
libsodium                 1.0.17               h01d97ff_0    conda-forge
libspatialite             4.3.0a            hb0905fd_1031    conda-forge
libssh2                   1.8.2                hcdc9a53_2    conda-forge
libtiff                   4.0.10            hd08fb8f_1003    conda-forge
libwebp                   1.0.2                h14d73da_2    conda-forge
libxcb                    1.13              h1de35cc_1002    conda-forge
libxml2                   2.9.9                h12c6b28_5    conda-forge
llvm-openmp               9.0.0                h40edb58_0    conda-forge
lz4-c                     1.8.3             h6de7cb9_1001    conda-forge
markupsafe                1.1.1            py37h1de35cc_0    conda-forge
matplotlib                3.1.1                    py37_1    conda-forge
matplotlib-base           3.1.1            py37h3a684a6_1    conda-forge
mccabe                    0.6.1                      py_1    conda-forge
mistune                   0.8.4           py37h1de35cc_1000    conda-forge
more-itertools            7.2.0                      py_0    conda-forge
nbconvert                 5.6.0                    py37_1    conda-forge
nbformat                  4.4.0                      py_1    conda-forge
nbsphinx                  0.4.3                      py_0    conda-forge
ncurses                   6.1               h0a44026_1002    conda-forge
netcdf4                   1.5.3            py37he9269ea_0    conda-forge
nettle                    3.4.1             h3efe00b_1002    conda-forge
nose                      1.3.7                 py37_1002    conda-forge
notebook                  6.0.1                    py37_0    conda-forge
nspr                      4.20              h0a44026_1000    conda-forge
nss                       3.47                 hcec2283_0    conda-forge
numpy                     1.17.3           py37hde6bac1_0    conda-forge
numpydoc                  0.9.1                      py_0    conda-forge
olefile                   0.46                       py_0    conda-forge
openh264                  1.8.0             hd9629dc_1000    conda-forge
openjpeg                  2.3.1                hdc36067_1    conda-forge
openssl                   1.1.1c               h01d97ff_0    conda-forge
packaging                 19.2                       py_0    conda-forge
pandas                    0.25.2           py37h4f17bb1_0    conda-forge
pandoc                    2.7.3                         0    conda-forge
pandocfilters             1.4.2                      py_1    conda-forge
pango                     1.42.4               h6691c8e_1    conda-forge
parso                     0.5.1                      py_0    conda-forge
pcre                      8.43                 h4a8c4bd_0    conda-forge
pexpect                   4.7.0                    py37_0    conda-forge
pickleshare               0.7.5                 py37_1000    conda-forge
pillow                    6.2.1            py37hb6f49c9_0    conda-forge
pip                       19.3.1                   py37_0    conda-forge
pixman                    0.38.0            h01d97ff_1003    conda-forge
pluggy                    0.13.0                   py37_0    conda-forge
poppler                   0.67.0               hd5eb092_7    conda-forge
poppler-data              0.4.9                         1    conda-forge
postgresql                11.5                 h25afefd_1    conda-forge
proj                      6.2.0                hca663eb_1    conda-forge
prometheus_client         0.7.1                      py_0    conda-forge
prompt_toolkit            2.0.10                     py_0    conda-forge
pthread-stubs             0.4               h1de35cc_1001    conda-forge
ptyprocess                0.6.0                   py_1001    conda-forge
py                        1.8.0                      py_0    conda-forge
pycodestyle               2.5.0                      py_0    conda-forge
pycparser                 2.19                     py37_1    conda-forge
pyflakes                  2.1.1                      py_0    conda-forge
pygments                  2.4.2                      py_0    conda-forge
pylint                    2.4.3                    py37_0    conda-forge
pyopenssl                 19.0.0                   py37_0    conda-forge
pyparsing                 2.4.2                      py_0    conda-forge
pyqt                      5.12.3           py37h2a560b1_0    conda-forge
pyqt5-sip                 4.19.18                  pypi_0    pypi
pyqtwebengine             5.12.1                   pypi_0    pypi
pyrsistent                0.15.4           py37h01d97ff_0    conda-forge
pysocks                   1.7.1                    py37_0    conda-forge
pytest                    5.2.1                    py37_0    conda-forge
pytest-cov                2.8.1                      py_0    conda-forge
pytest-mpl                0.10                       py_0    conda-forge
python                    3.7.3                h93065d6_1    conda-forge
python-dateutil           2.8.0                      py_0    conda-forge
pytz                      2019.3                     py_0    conda-forge
pyzmq                     18.1.0           py37hee98d25_0    conda-forge
qt                        5.12.5               h1b46049_0    conda-forge
qtconsole                 4.5.5                      py_0    conda-forge
readline                  8.0                  hcfe32e1_0    conda-forge
requests                  2.22.0                   py37_1    conda-forge
send2trash                1.5.0                      py_0    conda-forge
setuptools                41.4.0                   py37_0    conda-forge
six                       1.12.0                py37_1000    conda-forge
snowballstemmer           2.0.0                      py_0    conda-forge
sphinx                    1.8.5                    py37_0    conda-forge
sphinx-gallery            0.4.0                    py37_0    conda-forge
sphinx_rtd_theme          0.4.3                      py_0    conda-forge
sphinxcontrib-websupport  1.1.2                      py_0    conda-forge
sqlite                    3.30.1               h93121df_0    conda-forge
tbb                       2018.0.5             h2d50403_0    conda-forge
terminado                 0.8.2                    py37_0    conda-forge
testpath                  0.4.2                   py_1001    conda-forge
tiledb                    1.6.2                h4f44bfb_1    conda-forge
tk                        8.6.9             h2573ce8_1003    conda-forge
toml                      0.10.0                     py_0    conda-forge
tornado                   6.0.3            py37h01d97ff_0    conda-forge
traitlets                 4.3.3                    py37_0    conda-forge
tzcode                    2019a             h01d97ff_1002    conda-forge
urllib3                   1.25.6                   py37_0    conda-forge
wcwidth                   0.1.7                      py_1    conda-forge
webencodings              0.5.1                      py_1    conda-forge
wheel                     0.33.6                   py37_0    conda-forge
widgetsnbextension        3.5.1                    py37_0    conda-forge
wrapt                     1.11.2           py37h01d97ff_0    conda-forge
x264                      1!152.20180806       h1de35cc_0    conda-forge
xarray                    0.14.0                     py_0    conda-forge
xerces-c                  3.2.2             hbda6038_1004    conda-forge
xorg-kbproto              1.0.7             h1de35cc_1002    conda-forge
xorg-libice               1.0.10               h01d97ff_0    conda-forge
xorg-libsm                1.2.3             h01d97ff_1000    conda-forge
xorg-libx11               1.6.9                h0b31af3_0    conda-forge
xorg-libxau               1.0.9                h1de35cc_0    conda-forge
xorg-libxdmcp             1.1.3                h01d97ff_0    conda-forge
xorg-libxext              1.3.4                h01d97ff_0    conda-forge
xorg-xextproto            7.3.0             h1de35cc_1002    conda-forge
xorg-xproto               7.0.31            h1de35cc_1007    conda-forge
xz                        5.2.4             h1de35cc_1001    conda-forge
zeromq                    4.3.2                h6de7cb9_2    conda-forge
zipp                      0.6.0                      py_0    conda-forge
zlib                      1.2.11            h0b31af3_1006    conda-forge
zstd                      1.4.0                ha9f0a20_0    conda-forge
@weiji14
Copy link
Member

weiji14 commented Nov 2, 2019

Hmm, I've managed to reproduce this on my computer. Might be something to do with the virtualfile_from_grid mechanism, but I can't tell for sure. Will need to investigate.

@weiji14 weiji14 added the bug Something isn't working label Nov 2, 2019
@liamtoney
Copy link
Member Author

My current work-around is to generate an illumination grid via grdgradient and apply that to my grdimage call, but it's pretty clunky.

@liamtoney
Copy link
Member Author

Here it is, in all of its twice-nested-context-manager glory:

import pygmt

fig = pygmt.Figure()

dem = pygmt.datasets.load_earth_relief('10m')

with pygmt.helpers.GMTTempFile() as tmp_grd:
    with pygmt.clib.Session() as session:
        with session.virtualfile_from_grid(dem) as dem_file:
            session.call_module('grdgradient', f'{dem_file} -A-45 -Nt1- -G{tmp_grd.name}')
    fig.grdimage(dem, region=[-180, 180, -90, 90], frame='af',
                 projection='Cyl_stere/6i', cmap='geo', shading=tmp_grd.name)

fig.show(method='external')

@ibrewster
Copy link

@liamtoney For what it's worth, when I try your nested context manager"solution", I just get an error:

grdimage [ERROR]: Dimensions of intensity grid do not match that of the data grid!

Using the @ name, however, seems to work well - and runs MUCH faster, for some reason.

@liamtoney
Copy link
Member Author

@liamtoney For what it's worth, when I try your nested context manager"solution", I just get an error:

Are you running the exact same code as above? Perhaps using a different grid make yours not work? I just tested the above code again on the GMT Binder, and it worked fine. You can try it here:

Binder

@ibrewster
Copy link

Smaller grid (tried at 01m and 30s), smaller region, and different projection, but otherwise copied-and-pasted your code, yes.

Of course, this is still just a workaround for shading=True not working (though, in my case, it actually crashes Python. Maybe Python version difference? Hmmm...)

@liamtoney
Copy link
Member Author

@ibrewster I think it's an issue in regular GMT, see GenericMappingTools/gmt#3408

@seisman seisman added the upstream Bug or missing feature of upstream core GMT label Jun 21, 2020
@seisman
Copy link
Member

seisman commented Jun 21, 2020

It's confirmed to be a bug in GMT. Upstream PRs GenericMappingTools/gmt#3510 and GenericMappingTools/gmt#3512 fixed the bug and were already merged into GMT's master branch.

If you build the latest GMT master branch, running @liamtoney's example script in the top post will give you a beautiful earth relief map with shadings.

image

For users who don't want to manually build the GMT source codes, you can wait for the upcoming GMT 6.1.0, which is planned to be released before July 1st.


NOTE for PyGMT maintainers:

We should add a test for this after GMT 6.1.0 is released.

@liamtoney
Copy link
Member Author

I noticed another issue with shading, this time in grdview(), when making #502 — posting here because I think it's the same problem and should be fixed by the GMT upstream fix — @seisman could you confirm?


Here I construct the same grid using both grdmath (took me a bit!) and Python:

Pure GMT

gmt grdmath -R-5/5/-5/5 -I0.05 X Y HYPOT 0.5 0.5 POW MUL -0.2 MUL EXP -20 MUL 2 PI MUL X MUL COS 2 PI MUL Y MUL COS ADD 0.5 MUL EXP SUB E ADD 20 ADD = data.grd

Python

import numpy as np
import xarray

def ackley(x, y):
    return (
        -20 * np.exp(-0.2 * np.sqrt(0.5 * (x ** 2 + y ** 2)))
        - np.exp(0.5 * (np.cos(2 * np.pi * x) + np.cos(2 * np.pi * y)))
        + np.exp(1)
        + 20
    )

INC = 0.05
x = np.arange(-5, 5 + INC, INC)
y = np.arange(-5, 5 + INC, INC)
data = xarray.DataArray(ackley(*np.meshgrid(x, y)), coords=(x, y))

Then the following script, with INPUT = "data.grd", produces the correct result (at left) while INPUT = data produces an unexpected result (at right).

import pygmt
fig = pygmt.Figure()
SCALE = 0.2  # [inches]
fig.grdview(
    INPUT,
    frame=["a5f1", "za5f1"],
    projection=f"x{SCALE}i",
    zscale=f"{SCALE}i",
    surftype="s",
    cmap="roma",
    perspective="135/30",
    I="+",
)
fig.show()
INPUT = "data.grd" INPUT = data
good bad

@seisman
Copy link
Member

seisman commented Jul 1, 2020

@liamtoney It's a nice figure and a nice test! I can reproduce your issue with GMT 6.0.0, but CAN'T reproduce it with GMT master. I believe it's already fixed in upstream GMT.

@liamtoney
Copy link
Member Author

I believe it's already fixed in upstream GMT.

Awesome, thanks for testing. I can edit the script in #502 to add I="+" once we support newer GMT.

@weiji14 weiji14 added this to the 0.2.x milestone Jul 7, 2020
@liamtoney
Copy link
Member Author

liamtoney commented Jul 7, 2020

Confirmed that both of the problematic examples above are fixed using PyGMT 1.2.0 and (the actual fix) GMT 6.1.0!

@weiji14
Copy link
Member

weiji14 commented Jul 11, 2020

We should add a test for this after GMT 6.1.0 is released.

Ok, GMT 6.1.0 is released and we've bumped the master branch of PyGMT to use it (thanks @seisman!). If someone wants to learn a bit about writing unit tests, this might an easy one to have a go at.

@weiji14 weiji14 added the scipy-sprint Things to work on during the SciPy sprint! label Jul 11, 2020
@MarkWieczorek
Copy link
Contributor

I think that the docstring of grdimage needs to be updated. It is not clear what shading=True means (there is no similar option in gmt's grdimage). The default should be to provide an xarray grid, but this is not currently supported.

@liamtoney
Copy link
Member Author

If the shading kwarg is not provided, then no shading is performed. If it is provided as shading=True, I think it should default to +d — see the GMT info here: https://docs.generic-mapping-tools.org/latest/grdimage.html#i. Currently, shading=True is the equivalent of -I with no args. According to the linked grdimage docs the default for a bare -I is no illumination. However, trying a simple GMT grdimage one-liner results in shading even if the bare -I is used. I.e., -I and -I+d return the same shaded image. (Side note: The gmt grdimage docs for the -I flag could be more clear on this.)

In any case, I agree that the docstring for grdimage needs to be updated. The docstring for grdview (dev docs) is here: https://www.pygmt.org/dev/api/generated/pygmt.Figure.grdview.html#pygmt.Figure.grdview. We could use the same for grdimage and grdview, but add in (for both) that shading=True is equivalent to -I which (apparently) is equivalent to -I+d.

The default should be to provide an xarray grid, but this is not currently supported.

I don't understand what you mean by default in this context, but I do think this should be supported. Probably good to make a new feature request for this though.

@MarkWieczorek
Copy link
Contributor

For reference, here is what the gmt man page says about the -I parameter (the web documentation is worded slightly differently):

usage: gmt grdimage ... -I[<intensgrid>|<value>|<modifiers]

and

-I Apply directional illumination. Append name of intensity grid file. For a constant intensity (i.e., change the ambient light), append a value. To derive intensities from <grd_z> instead, append +a<azim> [-45], +n<method> [t1], and +m<ambient> [0] or use -I+d to accept the default values (see grdgradient for details). To derive from another grid than <grd_z>, give a filename as well.

Thus, option 1 is to use a gridfile, option 2 is for a constant value, and option 3 is to derive intensities using grdgradient. If you chose option 3, then +d uses the default values of option 3.

When the -I was first introduced, the only option was to specify a gridfile, and this usage persisted up until about gmt v5 when options 2 and 3 will introduced. This is why I just assumed that option 1 should be the default (i.e., I didn't know about the other 2 options until today!). However, +d might be a good fallback behaviour if -I is called with no other parameters.

@weiji14
Copy link
Member

weiji14 commented Oct 15, 2020

Looks like this got fixed in GMT upstream at GenericMappingTools/gmt#4328, I see the xarray shading test we added in #581 is now an xpass instead of xfail. So anyone who builds GMT from source (or wait for GMT 6.2.0 in Feb/Mar 2021?) should not face this bug anymore. In the meantime, I've made a PR at conda-forge/gmt-feedstock#108 so people can run conda install -c conda-forge/label/dev gmt to install a GMT development version to test things out now too 😁

seisman added a commit that referenced this issue Oct 15, 2020
The xarray shading issue in #364 was fixed by upstream GMT in GenericMappingTools/gmt#4328.

This PR updates the pytest xfail condition so that the xarray shading test is expected to xfail only for GMT<=6.1.1.
@seisman
Copy link
Member

seisman commented Oct 15, 2020

Closed by GMT upstream PR GenericMappingTools/gmt#4328.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working scipy-sprint Things to work on during the SciPy sprint! upstream Bug or missing feature of upstream core GMT
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants