Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark grdsample, grdfilter and sph2grd with fixed cores #2945

Merged
merged 9 commits into from
Jan 4, 2024
Merged

Benchmark grdsample, grdfilter and sph2grd with fixed cores #2945

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
a2a5901
Run test_grdsample_dataarray_out with 2 cores
weiji14 Jan 2, 2024
e402e6d
Run test_sph2grd_no_outgrid with 2 cores
weiji14 Jan 2, 2024
1f05247
Merge branch 'main' into benchmark-fixed-cores
weiji14 Jan 2, 2024
56d50be
Set GMT_MAX_CORES=2 in benchmarks.yml
weiji14 Jan 2, 2024
452308d
Merge branch 'main' into benchmark-fixed-cores
weiji14 Jan 4, 2024
56b47f2
Run test_grdimage_grid_and_shading_with_xarray with 2 cores
weiji14 Jan 4, 2024
d430d8e
Run test_grdfilter_dataarray_in_dataarray_out with 2 cores
weiji14 Jan 4, 2024
08f36d2
Revert "Run test_grdimage_grid_and_shading_with_xarray with 2 cores"
weiji14 Jan 4, 2024
31dce8e
Revert "Set GMT_MAX_CORES=2 in benchmarks.yml"
weiji14 Jan 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/benchmarks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ jobs:
with:
run: |
python -c "import pygmt; pygmt.show_versions()"
gmt set GMT_MAX_CORES=2
seisman marked this conversation as resolved.
Show resolved Hide resolved
PYGMT_USE_EXTERNAL_DISPLAY="false" python -m pytest -r P --pyargs pygmt --codspeed
env:
GMT_LIBRARY_PATH: /usr/share/miniconda/lib/
2 changes: 1 addition & 1 deletion pygmt/tests/test_grdfilter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ def test_grdfilter_dataarray_in_dataarray_out(grid, expected_grid):
Test grdfilter with an input DataArray, and output as DataArray.
"""
result = grdfilter(
grid=grid, filter="g600", distance="4", region=[-53, -49, -20, -17]
grid=grid, filter="g600", distance="4", region=[-53, -49, -20, -17], cores=2
)
# check information of the output grid
assert isinstance(result, xr.DataArray)
Expand Down
4 changes: 2 additions & 2 deletions pygmt/tests/test_grdimage.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,10 +139,10 @@ def test_grdimage_grid_and_shading_with_xarray(grid, xrgrid):
"""
fig_ref, fig_test = Figure(), Figure()
fig_ref.grdimage(
grid="@earth_relief_01d_g", region="GL", cmap="geo", shading=xrgrid
grid="@earth_relief_01d_g", region="GL", cmap="geo", shading=xrgrid, cores=2
)
Copy link
Member Author

@weiji14 weiji14 Jan 4, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is raising an error at https://github.com/GenericMappingTools/pygmt/actions/runs/7404780202/job/20146784435?pr=2945#step:8:778:

 E           pygmt.exceptions.GMTCLibError: Module 'grdimage' failed with status code 72:
Error:      grdimage [ERROR]: Option -x given more than once
Error:      grdimage [ERROR]: Option -x parsing failure. Correct syntax:
Error:      grdimage [ERROR]: Offending option -x2

Equivalent GMT CLI command also fails (both on GMT 6.4.0 and GMT 6.5.0.dev9+0776994):

gmt grdimage @earth_relief_01d_g -RGL -Cgeo -x2 -png map

grdimage [ERROR]: Option -x given more than once
grdimage [ERROR]: Option -x parsing failure. Correct syntax:

   [-x[[-]<n>]] .
     Limit the number of cores used in multi-threaded algorithms [Default uses all 16 cores]. If <n> is negative then we select (16 - <n>) cores (or at least 1).
grdimage [ERROR]: Offending option -x2

Might be an upstream GMT bug?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very likely

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, reported upstream at GenericMappingTools/gmt#8256. I've reverted the grdimage commit at 08f36d2 for now.

fig_ref.colorbar()
fig_test.grdimage(grid=grid, region="GL", cmap="geo", shading=xrgrid)
fig_test.grdimage(grid=grid, region="GL", cmap="geo", shading=xrgrid, cores=2)
fig_test.colorbar()
return fig_ref, fig_test

Expand Down
2 changes: 1 addition & 1 deletion pygmt/tests/test_grdsample.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ def test_grdsample_dataarray_out(grid, expected_grid, region, spacing):
"""
Test grdsample with no outgrid set and the spacing is changed.
"""
result = grdsample(grid=grid, spacing=spacing, region=region)
result = grdsample(grid=grid, spacing=spacing, region=region, cores=2)
# check information of the output grid
assert isinstance(result, xr.DataArray)
assert result.gmt.gtype == 1 # Geographic grid
Expand Down
2 changes: 1 addition & 1 deletion pygmt/tests/test_sph2grd.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ def test_sph2grd_no_outgrid():
"""
Test sph2grd with no set outgrid.
"""
temp_grid = sph2grd(data="@EGM96_to_36.txt", spacing=1, region="g")
temp_grid = sph2grd(data="@EGM96_to_36.txt", spacing=1, region="g", cores=2)
assert temp_grid.dims == ("y", "x")
assert temp_grid.gmt.gtype == 0 # Cartesian grid
assert temp_grid.gmt.registration == 0 # Gridline registration
Expand Down
Loading