Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix UnicodeDecodeError with shapefiles for plot and plot3d #1695

Merged
merged 14 commits into from
Mar 1, 2022
Merged

Fix UnicodeDecodeError with shapefiles for plot and plot3d #1695

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
cc82804
Fix UnicodeDecodeError with shapefiles for plot and plot3d
seisman Jan 6, 2022
2d051db
Merge branch 'main' into ogrgmt-detection
seisman Jan 7, 2022
b85269d
Merge branch 'main' into ogrgmt-detection
seisman Feb 24, 2022
6edca91
Add a new test for plotting shapefiles
seisman Feb 24, 2022
f2873e2
Merge branch 'main' into ogrgmt-detection
seisman Feb 27, 2022
f7f9286
Remove data files and use caches from the GMT server
seisman Feb 27, 2022
a461183
Cache files
seisman Feb 27, 2022
4c6e3c0
Enable data cache workflow in PR
seisman Feb 27, 2022
47ea523
Remove RidgeTest.gmt
seisman Feb 27, 2022
67c58e5
Disable cache data workflow in PR
seisman Feb 27, 2022
5f7250d
Merge branch 'main' into ogrgmt-detection
seisman Feb 28, 2022
b4e2583
Fix formatting
seisman Feb 28, 2022
8c617b3
Trigger data cache in PR
seisman Mar 1, 2022
3712edc
Disable data cache in PR
seisman Mar 1, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 4 additions & 6 deletions pygmt/src/plot.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -229,15 +229,13 @@ def plot(self, data=None, x=None, y=None, size=None, direction=None, **kwargs):
and data.geom_type.isin(["Point", "MultiPoint"]).all()
): # checking if the geometry of a geoDataFrame is Point or MultiPoint
kwargs["S"] = "s0.2c"
elif (
"S" not in kwargs and kind == "file"
): # checking that the data is a file path to set default style
elif "S" not in kwargs and kind == "file" and data.endswith(".gmt"):
# checking that the data is a file path to set default style
try:
with open(which(data), mode="r", encoding="utf8") as file:
line = file.readline()
if (
"@GMULTIPOINT" in line or "@GPOINT" in line
): # if the file is gmt style and geometry is set to Point
if "@GMULTIPOINT" in line or "@GPOINT" in line:
# if the file is gmt style and geometry is set to Point
kwargs["S"] = "s0.2c"
except FileNotFoundError:
pass
Expand Down
10 changes: 4 additions & 6 deletions pygmt/src/plot3d.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,15 +199,13 @@ def plot3d(
and data.geom_type.isin(["Point", "MultiPoint"]).all()
): # checking if the geometry of a geoDataFrame is Point or MultiPoint
kwargs["S"] = "u0.2c"
elif (
"S" not in kwargs and kind == "file"
): # checking that the data is a file path to set default style
elif "S" not in kwargs and kind == "file" and data.endswith(".gmt"):
# checking that the data is a file path to set default style
try:
with open(which(data), mode="r", encoding="utf8") as file:
line = file.readline()
if (
"@GMULTIPOINT" in line or "@GPOINT" in line
): # if the file is gmt style and geometry is set to Point
if "@GMULTIPOINT" in line or "@GPOINT" in line:
# if the file is gmt style and geometry is set to Point
kwargs["S"] = "u0.2c"
except FileNotFoundError:
pass
Expand Down
4 changes: 4 additions & 0 deletions pygmt/tests/baseline/test_plot_shapefile.png.dvc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
outs:
- md5: 75277741d098cf7a0bad7869b574afc9
size: 24178
path: test_plot_shapefile.png
Binary file added pygmt/tests/data/RidgeTest.dbf
View file
Open in desktop
Binary file not shown.
105 changes: 105 additions & 0 deletions pygmt/tests/data/RidgeTest.gmt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# @VGMT1.0 @GLINESTRING
# @R-39.5848/-28.3229/33.0656/44.3014
# @Je4326
# @Jp"+proj=longlat +datum=WGS84 +no_defs "
# @Jw"GEOGCS[\"WGS 84\",DATUM[\"WGS_1984\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],AUTHORITY[\"EPSG\",\"6326\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4326\"]]"
# @NPLATEID1|GPGIM_TYPE|TYPE|FROMAGE|TOAGE|NAME|DESCR|FEATURE_ID|PLATEID2|L_PLATE|R_PLATE|RECON_METH|SPREAD_ASY|REGION_NO|REF_NO|STRING_NO|TYPE_NO|TYPE_NO_AD|COLOUR|NPOINTS
# @Tdouble|string|string|double|double|string|string|string|double|double|double|string|double|double|double|double|double|string|double|double
# FEATURE_DATA
>
# @D101|gpml:MidOceanRidge|RI|0.000000000000000|-999.000000000000000|"Reykjanes"||GPlates-1389768d-90d9-42e8-95fa-5b441c7fb297|304|0|0||0.000000000000000|92|1|7|1||5|57
-28.3229 44.3014
-28.371 44.1811
-28.4257 44.199
-28.4434 44.1443
-28.4463 44.0954
-28.4735 44.0269
-28.4886 43.9327
-28.5103 43.8392
-28.5286 43.7624
-28.7955 43.7719
-28.8334 43.6971
-28.8841 43.626
-28.9364 43.4911
-29.0063 43.3874
-29.0994 43.2097
-29.1737 43.077
-29.3101 43.0849
-29.3108 43.0237
-29.3264 42.9539
-29.3505 42.8359
-29.3758 42.7278
-29.3147 42.7148
-29.3218 42.6737
-29.3402 42.6067
-29.36 42.53
-29.3897 42.415
-29.4283 42.2861
-29.2223 42.2888
-29.2285 42.233
-29.2442 42.1805
-29.2445 42.0824
-29.2612 41.9269
-29.1583 41.9123
-29.1967 41.803
-29.2144 41.6796
-29.247 41.5257
-29.2837 41.3845
-29.3199 41.2457
-29.2247 41.2098
-29.2391 41.1228
-29.2591 40.9825
-29.2778 40.869
-29.3081 40.7027
-29.3399 40.5659
-29.5671 40.5991
-29.5711 40.5529
-29.5749 40.4674
-29.5949 40.3809
-29.6282 40.2908
-29.6382 40.2059
-29.6535 40.1068
-29.58 40.1051
-29.588 40.0739
-29.6102 40.0342
-29.6087 39.9929
-29.762 39.3973
-29.9062 39.3866
>
# @D101|gpml:MidOceanRidge|RI|0.000000000000000|-999.000000000000000|"Klitgord"||GPlates-53da72c0-30ba-4fee-9240-df8468b1c915|714|0|0||0.000000000000000|91|1|10|1||5|20
-29.9062 39.3866
-30.4279 38.387
-30.6063 38.3996
-31.0731 37.6793
-31.3683 37.7048
-31.5147 37.2837
-32.2698 37.375
-32.2971 37.4118
-32.3909 37.1394
-32.6448 37.176
-32.7066 37.0349
-32.9468 37.0643
-33.0039 36.9175
-33.2235 36.9432
-33.3173 36.6931
-33.5941 36.741
-33.7725 36.1848
-33.9852 36.22
-34.0539 36.0716
-34.174 36.103
>
# @D101|gpml:MidOceanRidge|RI|0.000000000000000|-999.000000000000000|"Dietmar"||GPlates-0067db63-b172-42e9-9020-fbca230de033|714|0|0||0.000000000000000|91|1|11|1||5|14
-34.174 36.103
-34.294 35.736
-34.414 35.736
-34.745 35.146
-36.336 35.368
-36.637 34.751
-36.787 34.751
-36.877 34.552
-37.057 34.577
-37.237 34.154
-37.357 34.179
-37.628 33.603
-39.279 33.879
-39.5848 33.0656
1 change: 1 addition & 0 deletions pygmt/tests/data/RidgeTest.prj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137.0,298.257223563]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]]
Copy link
Member

@weiji14 weiji14 Feb 24, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still not sure if storing the shapefile (shp/shx/dbf/prj) in the git history is a good idea. I'm thinking whether we should put it on dvc, or maybe add some custom code to pygmt.helpers.testing.download_test_data() to download the shapefile using urllib or plain pygmt.which.

from pygmt import which

for ext in ["shp", "shx", "dbf", "prj"]:
    which(
        fname=f"https://github.com/intake/intake_geopandas/raw/0.4.0/tests/data/stations/stations.{ext}",
        download=True,
    )

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option would be to convert RidgeTest.gmt to a shapefile (possibly we could cache the shapefile to speed up the new PyGMT test and https://github.com/GenericMappingTools/gmt/blob/master/test/psxy/categorical.sh):

test_data=`gmt which -Gl @RidgeTest.gmt`
ogr2ogr -f "ESRI Shapefile" RidgeTest.shp $test_data

I'm considering the option from @meghanrjones again. I think it makes more sense to cache the shapefile in the GMT server, because they can also be used by GMT and GMT.jl.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened a PR in GenericMappingTools/gmtserver-admin#142.

Binary file added pygmt/tests/data/RidgeTest.shp
View file
Open in desktop
Binary file not shown.
Binary file added pygmt/tests/data/RidgeTest.shx
View file
Open in desktop
Binary file not shown.
12 changes: 12 additions & 0 deletions pygmt/tests/test_plot.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -544,3 +544,15 @@ def test_plot_ogrgmt_file_multipoint_non_default_style():
style="c0.2c",
)
return fig


@pytest.mark.mpl_image_compare
def test_plot_shapefile():
"""
Make sure that plot works for shape file.

See https://github.com/GenericMappingTools/pygmt/issues/1616.
"""
fig = Figure()
fig.plot(data=os.path.join(TEST_DATA_DIR, "RidgeTest.shp"), pen="1p")
return fig