Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

r.in.pdal: Improve PDAL error handling #4750

Merged
merged 6 commits into from
Dec 2, 2024
Merged

Conversation

marisn
Copy link
Contributor

@marisn marisn commented Nov 24, 2024

Some non-conformant LAS files trigger an error in PDAL library that results in an unhandled exit from r.in.pdal module. This code catches all PDAL errors and reports them in a GRASS way.

One of error sources is lack of SRS information in the imported LAS file. This code propagates SRS check override from GRASS to PDAL library thus eliminating an error observed by non-conformant 1.4 LAS files.

Fixes: #4157

See also:
qgis/QGIS#51862 (comment)
PDAL/PDAL#3818

Some non-conformant LAS files trigger an error in PDAL library that
results in an unhandled exit from r.in.pdal module.
This code catches all PDAL errors and reports them in a GRASS way.

One of error sources is lack of SRS information in the imported LAS file.
This code propogates SRS check override from GRASS to PDAL library
thus eliminating an error observed by non-conformant 1.4 LAS files.

Fixes: OSGeo#4157
@marisn marisn added raster Related to raster data processing C++ Related code is in C++ backport to 8.4 PR needs to be backported to release branch 8.4 labels Nov 24, 2024
@marisn marisn added this to the 8.5.0 milestone Nov 24, 2024
@marisn marisn requested a review from wenzeslaus November 24, 2024 12:55
@github-actions github-actions bot added Python Related code is in Python module tests Related to Test Suite labels Nov 24, 2024
@echoix
Copy link
Member

echoix commented Nov 24, 2024

Seems an appropriate solution to send nosrs to PDAL by reading your linked threads. Was the "nosrs" option available for all our supported versions? If not, we might need to change our version requirements.

@echoix
Copy link
Member

echoix commented Nov 24, 2024

@marisn I didn't know you were the person of knowledge on that topic. In the QGIS thread linked, it suggested having PDAL >=2.7.2. We had a problem in our docker images, where the PDAL version couldn't be upgraded, and we reverted it. #3960
Do you have an idea how we could get it unblocked? The later versions enforced a proj version that wasn't available for the ubuntu repositories.

One of the docker images is stuck on 2.7.1 (ubuntu wxGUI), and the debian one on 2.7.2.

@marisn
Copy link
Contributor Author

marisn commented Nov 24, 2024

Seems an appropriate solution to send nosrs to PDAL by reading your linked threads. Was the "nosrs" option available for all our supported versions? If not, we might need to change our version requirements.

Good catch! It has been added in PDAL 2.3.0 – we'll have to come up with some version check as PDAL doesn't like unknown options. Our minimum PDAL version is 1.7.1, although I have no idea if it is even possible to compile GRASS with all of its dependencies to work together with such ancient version.

@marisn marisn marked this pull request as draft November 24, 2024 13:10
@echoix
Copy link
Member

echoix commented Nov 24, 2024

Seems an appropriate solution to send nosrs to PDAL by reading your linked threads. Was the "nosrs" option available for all our supported versions? If not, we might need to change our version requirements.

Good catch! It has been added in PDAL 2.3.0 – we'll have to come up with some version check as PDAL doesn't like unknown options. Our minimum PDAL version is 1.7.1, although I have no idea if it is even possible to compile GRASS with all of its dependencies to work together with such ancient version.

We don't need to plan supporting such incompatible versions. Just like the Linux kernel assumes that, at one point, some older hardware isn't expected to be running the latest and greatest. It is normal for software to increase the requirements range at each release.

@marisn
Copy link
Contributor Author

marisn commented Nov 28, 2024

I changed the configure.ac file to check for PDAL version as I was unable to find a solution how to check if PDAL reader.las has a "nosrs" as a valid option. I tested the configure part and module with PDAL 1.8.0, 2.4.2, 2.4.3, 2.5.0 and current master.

Only problem remaining is a need to regenerate files created with autoheader and autoconf – when I run those commands on my system, I get a huge list of changes in their generated files in comparison to current ones in the git. Hints how to tackle it?

@marisn
Copy link
Contributor Author

marisn commented Nov 28, 2024

We don't need to plan supporting such incompatible versions.

readers.las "nosrs" option makes sense only since 2.4.3 (if I read correctly PDAL/PDAL#3818), but that is not available even in Ubuntu 22.04. The "nosrs" option does not cause runtime error only since 2.3.0, thus would not work with Debian 10 and 11 (there is no official PDAL package in 12). Thus al though the most recent distributions are fine, PDAL <2.4.3 is not that uncommon yet.

@nilason
Copy link
Contributor

nilason commented Nov 28, 2024

I changed the configure.ac file to check for PDAL version as I was unable to find a solution how to check if PDAL reader.las has a "nosrs" as a valid option. I tested the configure part and module with PDAL 1.8.0, 2.4.2, 2.4.3, 2.5.0 and current master.

Would it be possible to use the macros PDAL_VERSION_MAJOR, PDAL_VERSION_MINOR and PDAL_VERSION_PATCH defined in pdal_features.hpp, instead of complicating configure?

Only problem remaining is a need to regenerate files created with autoheader and autoconf – when I run those commands on my system, I get a huge list of changes in their generated files in comparison to current ones in the git. Hints how to tackle it?

What is the version of your autoconf?

Reduces overhead of the configure stage
@marisn
Copy link
Contributor Author

marisn commented Nov 28, 2024

Would it be possible to use the macros PDAL_VERSION_MAJOR, PDAL_VERSION_MINOR and PDAL_VERSION_PATCH defined in pdal_features.hpp, instead of complicating configure?

You are right. This is a better approach. I was so focused on having a C++ code that performs the check during configure stage (and failing miserably to come up with one), that I missed the most obvious solution. Thanks!

What is the version of your autoconf?

autoconf (GNU Autoconf) 2.72
I rolled back files touched by auto tools and thus this is just a theoretical problem now.

@marisn
Copy link
Contributor Author

marisn commented Nov 28, 2024

Would it be possible to use the macros PDAL_VERSION_MAJOR, PDAL_VERSION_MINOR and PDAL_VERSION_PATCH defined in pdal_features.hpp, instead of complicating configure?

We could also replace existing check for NoFilenameWriter in the configure stage with version check in the header file. This would speed up our CI by a small amount.

@nilason
Copy link
Contributor

nilason commented Nov 28, 2024

Would it be possible to use the macros PDAL_VERSION_MAJOR, PDAL_VERSION_MINOR and PDAL_VERSION_PATCH defined in pdal_features.hpp, instead of complicating configure?

You are right. This is a better approach. I was so focused on having a C++ code that performs the check during configure stage (and failing miserably to come up with one), that I missed the most obvious solution. Thanks!

Great, looks much better!

What is the version of your autoconf?

autoconf (GNU Autoconf) 2.72 I rolled back files touched by auto tools and thus this is just a theoretical problem now.

We're still at 2.71, that's why all the unrelated changes.

@echoix
Copy link
Member

echoix commented Nov 28, 2024

It's nice to avoid having to port the autoconf/configure changes to cmake

@marisn marisn marked this pull request as ready for review November 28, 2024 14:57
@echoix echoix merged commit 856a514 into OSGeo:main Dec 2, 2024
26 checks passed
@neteler neteler modified the milestones: 8.5.0, 8.4.1 Dec 18, 2024
@neteler neteler removed the backport to 8.4 PR needs to be backported to release branch 8.4 label Dec 18, 2024
neteler pushed a commit to neteler/grass that referenced this pull request Dec 18, 2024
* r.in.pdal: Improve PDAL error handling
Some non-conformant LAS files trigger an error in PDAL library that
results in an unhandled exit from r.in.pdal module.
This code catches all PDAL errors and reports them in a GRASS way.

One of error sources is lack of SRS information in the imported LAS file.
This code propogates SRS check override from GRASS to PDAL library
thus eliminating an error observed by non-conformant 1.4 LAS files.

Fixes: OSGeo#4157

* r.in.pdal: update copyright notice

* r.in.pdal: enable reader.las "nosrs" option only in PDAL >= 2.4.3

* r.in.pdal: use PDAL defined macros to determine its version
Reduces overhead of the configure stage

* r.in.pdal: remove debugging pritnout left by accident

* r.in.pdal: do not pass unused parameter to functions if "nosrs" is
not available
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C++ Related code is in C++ libraries module Python Related code is in Python raster Related to raster data processing tests Related to Test Suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

r.in.pdal fails with LAS files without projection
4 participants