Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use fid in stead of nen3610id as index of dataframes in datamodel #17

Closed
d2hydro opened this issue Feb 5, 2024 · 0 comments
Closed

Use fid in stead of nen3610id as index of dataframes in datamodel #17

d2hydro opened this issue Feb 5, 2024 · 0 comments
Labels
enhancement New feature or request
Milestone
Versie 1.3

Comments

@d2hydro
Copy link
Contributor

d2hydro commented Feb 5, 2024 •
edited
Loading

On the same dataset as used in #14, we'll encounter an issue in rule hydroobject id 2: Hydroobject kruist zichzelf alleen in combinatie met een duikersifonhevel of aquaduct

This raises an exception The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() on:

HyDAMOValidatieModule/hydamo_validation/functions/topologic.py

Line 275 in a1b0980

indices = [i for i in indices if i != gdf.index.get_loc(row.name)]

This Exception is triggered by the following sequence:

  • users-data contains duplicate values in nen3610id column (e.g NL.WBHCODE.27.hydroobject.317366)
  • duplicate data is NaN-ned at syntax-validation, so NL.WBHCODE.27.hydroobject.317366 is changed to NaN
  • nen3610id is then used at index of dataframes in the datamodel; in this code gdf
  • gdf.index.get_loc(NaN) returns bogus

image

Sollution; use original fid in GeoPackage as index in dataframes of the datamodel to avoid any non-unique or missing indices.
@d2hydro d2hydro self-assigned this Feb 5, 2024
@d2hydro d2hydro added this to the Versie 1.3 milestone Feb 5, 2024
@d2hydro d2hydro changed the title The truth value of an array with more than one element is ambiguous Use fid in stead of nen3610id as index of dataframes in datamodel Feb 5, 2024
@d2hydro d2hydro added the enhancement New feature or request label Feb 5, 2024
@d2hydro d2hydro removed their assignment Mar 2, 2024
DanielTollenaar added a commit to DanielTollenaar/HyDAMOValidatieModule that referenced this issue Apr 5, 2024
@DanielTollenaar
test pyogrio met use_fid_as_index
b0c0ec8 
HetWaterschapshuis#17
This was referenced Apr 5, 2024
Merged
Merged
DanielTollenaar added a commit that referenced this issue Jun 20, 2024
@DanielTollenaar
Version 1.3 (#15)
ee9ea1e 
* specified python and package versions / fixed tests

* misc improvements

* betere geometrie handling

* fixed tests

* validationrules 1.2 & ruff

added rules_1.2.json

ruff formatted:
- datasets.py
- validator.py

* refactor tests + test for validation_rules

* Improve `not_overlapping`

* verbeterde not_overlapping

Test-resultaten zoals in #11

* reformatted logical_validation.py

* extend not_overlapping

Extend `not_overlapping`, fixes: #10, #11 and #12

* Update ValidationRules_1.2.json

* Update validator.py

* fix schemas_path en test_1877

* Update .gitignore

* __version__ updated

* requirements updated

* Update __init__.py

* updated environment

fixes #16

* snap_to_branches

fixes #14

* test pyogrio met use_fid_as_index

#17

* add log to results

#19

* do not add empty gdf to datamodel

* hydamo versie 2.3

* misc improvements

* fix test_productie

* for build

* Update dev_environment.yml

* HYV-214

* close validator.log

* for testing

* fix niet-uitgevoerde (topo)logische validaties

* Update __init__.py

---------

Co-authored-by: Daniel <daniel@d2hydro.nl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
Versie 1.3
Development

No branches or pull requests
1 participant
@DanielTollenaar