Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

geo_point runtime field implementation #63164

Merged
merged 15 commits into from
Oct 21, 2020
Merged

Conversation

iverase
Copy link
Contributor

@iverase iverase commented Oct 2, 2020

Run time field that emits geo points from lat/lon values. For example, given a mapping like:

{
    "mappings": {
        "properties": {
            "location": {
                "lat": {"type" : "double"},
                "lon" : {"type" "double"}
            }
        }
    }
}

A geo_point runtime field can be declared as:

{
    "runtime_geo_point": {
        "type" : "runtime",
         "runtime_type": "geo_point",
         "script": {
             "source" : "emit(doc['location'].lat, doc['location'].lon);"
         }
}

where the emit function takes two arguments, the first argument is the latitude and the second argument the longitude.

@iverase iverase added >enhancement :Analytics/Geo Indexing, search aggregations of geo points and shapes :Search/Search Search-related issues that do not fall into other categories labels Oct 2, 2020
@iverase iverase requested review from nik9000 and javanna October 2, 2020 09:12
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (:Analytics/Geo)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (:Search/Search)

@elasticmachine elasticmachine added Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:Search Meta label for search team labels Oct 2, 2020
@iverase iverase marked this pull request as draft October 2, 2020 09:15
@iverase
Copy link
Contributor Author

iverase commented Oct 5, 2020

@elasticmachine run elasticsearch-ci/packaging-sample-windows

@iverase
Copy link
Contributor Author

iverase commented Oct 5, 2020

@elasticmachine run elasticsearch-ci/1

Copy link
Member

@nik9000 nik9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a look and it looks pretty reasonable.

return new GeoPointScriptFieldGeoShapeQuery(script, leafFactory(context), fieldName, geometries);
}

private static LatLonGeometry[] toLuceneGeometry(String name, QueryShardContext context, Geometry geometry) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this something we can move into core? It looks like something we'd have there already?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thats is my plan but it seems more difficult that it looks at first sight as every time we need to map a geometry to a Lucene geometry, it seems to be slightly different. As part of that effort I opened https://issues.apache.org/jira/browse/LUCENE-9552 but I didn't have much support so far. I try to move it.

@@ -67,6 +67,14 @@ protected static QueryShardContext mockContext(boolean allowExpensiveQueries) {
return mockContext(allowExpensiveQueries, null);
}

protected boolean supportsTermQueries() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it'd be better to make a new superclass that doesn't have these methods?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that we already have AbstractNonTextScriptFieldTypeTestCase that extends this class. Those methods are worthy running so adding a superclass you will miss that. My feeling is that we need an abstract class for every type of query and field types will implement all the ones they support?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So instead of an abstract superclass for the type of field it is per query? I could see that being better, maybe. I'm ok with this as is if we plan to refactor it soon enough anyway.

@iverase
Copy link
Contributor Author

iverase commented Oct 5, 2020

@elasticmachine run elasticsearch-ci/packaging-sample-windows

Copy link
Member

@nik9000 nik9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks entirely sensible to me.

@iverase iverase marked this pull request as ready for review October 8, 2020 12:27
@iverase
Copy link
Contributor Author

iverase commented Oct 8, 2020

Moved the PR to ready for review (although thanks Nik, you've done all the work already).

Copy link
Member

@javanna javanna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks @iverase . One meta ask is if you could update the description of the PR to include how you can define a geo point runtime field in the mappings, especially around the fact that emit in this case takes two arguments.

@nik9000 would you like to have one last look?

Copy link
Member

@nik9000 nik9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with it! I feel like I should find something worth changing because runtime fields are new and we don't have super established patterns, but I think this is the right things to do.

@iverase iverase merged commit 265a214 into elastic:master Oct 21, 2020
iverase added a commit to iverase/elasticsearch that referenced this pull request Oct 21, 2020
Run time field that emits geo points from lat/lon values.
iverase added a commit that referenced this pull request Oct 21, 2020
Run time field that emits geo points from lat/lon values.
@iverase iverase deleted the runtime_geopoint branch October 21, 2020 10:06
pugnascotia pushed a commit to pugnascotia/elasticsearch that referenced this pull request Oct 21, 2020
Run time field that emits geo points from lat/lon values.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/Geo Indexing, search aggregations of geo points and shapes >enhancement :Search/Search Search-related issues that do not fall into other categories Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:Search Meta label for search team v7.11.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants