propose run udf on vector cube #200
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
soxofaan
reviewed
Jun 5, 2023
| context: Optional[dict] = None, | ||
| env: EvalEnv, | ||
| ) -> "DriverVectorCube": | ||
| if dimension == "bands" and target_dimension == None and len(process) == 1 and next(iter(process.values())).get('process_id') == 'run_udf': |
There was a problem hiding this comment.
Shouldn't this check for the "geometries" dimension instead of the "bands" dimension?
There was a problem hiding this comment.
I thought so too at first, but when you want to run a udf for each feature, and get all its properties, then the dimension you are working on is "bands" I guess?
| from openeo.udf import FeatureCollection, UdfData | ||
| collection = FeatureCollection(id='VectorCollection', data=self._as_geopandas_df()) | ||
| data = UdfData( | ||
| proj={"EPSG": self._geometries.crs.to_epsg()}, feature_collection_list=[collection], user_context=context |
There was a problem hiding this comment.
only the vector data (labels along geometry dimension) is passed here as feature collection, but the vector cube might also contain actual data cube values
There was a problem hiding this comment.
Looking at _as_geopandas_df, I have the impression that it does merge the geometries with the datacube values?
I'm only a bit worried about from_geojson, which does not seem to extract any property values into cube values, but that's maybe not relevant to what I try to solve here.
soxofaan
reviewed
Jun 12, 2023
| from openeo_driver.datastructs import SarBackscatterArgs, ResolutionMergeArgs, StacAsset | ||
| from openeo_driver.errors import FeatureUnsupportedException, InternalException | ||
| from openeo_driver.util.geometry import GeometryBufferer, validate_geojson_coordinates | ||
| from openeo_driver.util.ioformats import IOFORMATS | ||
| from openeo_driver.util.utm import area_in_square_meters | ||
| from openeo_driver.utils import EvalEnv | ||
| from openeogeotrellis.backend import SingleNodeUDFProcessGraphVisitor |
There was a problem hiding this comment.
I was trying to build unit test for this feature: but failed on this:
import of geopyspark-driver in python driver code is not going to work
soxofaan
added a commit
that referenced
this pull request
Aug 2, 2023
soxofaan
added a commit
that referenced
this pull request
Aug 2, 2023
soxofaan
added a commit
that referenced
this pull request
Aug 2, 2023
soxofaan
added a commit
that referenced
this pull request
Aug 2, 2023
soxofaan
added a commit
that referenced
this pull request
Aug 2, 2023
- flattening: move options to export phase iso vector cube constructor - introduce `VectorCube.from_geodataframe` wiith support for promoting selected columns to cube values - regardless of promotion: all properties are still associated with `VectorCube.geometries` for now (otherwise properties can not be preserved when using `aggregate_spatial`, see Open-EO/openeo-api#504) - only promote numerical values by default for now
soxofaan
added a commit
that referenced
this pull request
Aug 3, 2023
soxofaan
added a commit
that referenced
this pull request
Aug 3, 2023
soxofaan
added a commit
that referenced
this pull request
Aug 3, 2023
- flattening: move options to export phase iso vector cube constructor - introduce `VectorCube.from_geodataframe` wiith support for promoting selected columns to cube values - regardless of promotion: all properties are still associated with `VectorCube.geometries` for now (otherwise properties can not be preserved when using `aggregate_spatial`, see Open-EO/openeo-api#504) - only promote numerical values by default for now
soxofaan
added a commit
that referenced
this pull request
Aug 3, 2023
soxofaan
added a commit
that referenced
this pull request
Aug 3, 2023
soxofaan
added a commit
that referenced
this pull request
Aug 3, 2023
- flattening: move options to export phase iso vector cube constructor - introduce `VectorCube.from_geodataframe` wiith support for promoting selected columns to cube values - regardless of promotion: all properties are still associated with `VectorCube.geometries` for now (otherwise properties can not be preserved when using `aggregate_spatial`, see Open-EO/openeo-api#504) - only promote numerical values by default for now
soxofaan
added a commit
that referenced
this pull request
Aug 3, 2023
aligns better with load_geojson spec
soxofaan
added a commit
that referenced
this pull request
Aug 3, 2023
soxofaan
added a commit
that referenced
this pull request
Aug 3, 2023
aligns better with load_geojson spec
soxofaan
added a commit
that referenced
this pull request
Aug 3, 2023
soxofaan
added a commit
that referenced
this pull request
Aug 3, 2023
soxofaan
added a commit
that referenced
this pull request
Aug 3, 2023
|
I iterated further on this feature branch and merged in f83d1cf |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Open-EO/openeo-geopyspark-driver#437