Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add country_name to the default properties of geoip ingest processor #62915

Merged
merged 3 commits into from
Sep 30, 2020
Merged

add country_name to the default properties of geoip ingest processor #62915

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
243666d
add country_name to the default properties of geoip ingest processor
gaobinlong Sep 25, 2020
a56de91
Merge remote-tracking branch 'origin/master' into geoip_patch
gaobinlong Sep 30, 2020
ef16ce4
add country_name field to the test result
gaobinlong Sep 30, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions docs/reference/ingest/processors/geoip.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ uncompressed. The `ingest-geoip` config directory is located at `$ES_CONFIG/inge
| `field` | yes | - | The field to get the ip address from for the geographical lookup.
| `target_field` | no | geoip | The field that will hold the geographical information looked up from the Maxmind database.
| `database_file` | no | GeoLite2-City.mmdb | The database filename in the geoip config directory. The ingest-geoip module ships with the GeoLite2-City.mmdb, GeoLite2-Country.mmdb and GeoLite2-ASN.mmdb files.
| `properties` | no | [`continent_name`, `country_iso_code`, `region_iso_code`, `region_name`, `city_name`, `location`] * | Controls what properties are added to the `target_field` based on the geoip lookup.
| `properties` | no | [`continent_name`, `country_iso_code`, `country_name`, `region_iso_code`, `region_name`, `city_name`, `location`] * | Controls what properties are added to the `target_field` based on the geoip lookup.
| `ignore_missing` | no | `false` | If `true` and `field` does not exist, the processor quietly exits without modifying the document
| `first_only` | no | `true` | If `true` only first found geoip data will be returned, even if `field` contains array
|======
Expand Down Expand Up @@ -81,6 +81,7 @@ Which returns:
"ip": "8.8.8.8",
"geoip": {
"continent_name": "North America",
"country_name": "United States",
"country_iso_code": "US",
"location": { "lat": 37.751, "lon": -97.822 }
}
Expand All @@ -90,7 +91,7 @@ Which returns:
// TESTRESPONSE[s/"_seq_no": \d+/"_seq_no" : $body._seq_no/ s/"_primary_term":1/"_primary_term" : $body._primary_term/]

Here is an example that uses the default country database and adds the
geographical information to the `geo` field based on the `ip` field`. Note that
geographical information to the `geo` field based on the `ip` field. Note that
this database is included in the module. So this:

[source,console]
Expand Down Expand Up @@ -130,6 +131,7 @@ returns this:
"ip": "8.8.8.8",
"geo": {
"continent_name": "North America",
"country_name": "United States",
"country_iso_code": "US",
}
}
Expand Down Expand Up @@ -276,6 +278,7 @@ GET /my_ip_locations/_search
"_source" : {
"geoip" : {
"continent_name" : "North America",
"country_name" : "United States",
"country_iso_code" : "US",
"location" : {
"lon" : -97.822,
Expand Down
4 changes: 2 additions & 2 deletions modules/ingest-geoip/src/main/java/org/elasticsearch/ingest/geoip/GeoIpProcessor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -376,11 +376,11 @@ private Map<String, Object> retrieveAsnGeoData(InetAddress ipAddress) {

public static final class Factory implements Processor.Factory {
static final Set<Property> DEFAULT_CITY_PROPERTIES = Collections.unmodifiableSet(EnumSet.of(
Property.CONTINENT_NAME, Property.COUNTRY_ISO_CODE, Property.REGION_ISO_CODE,
Property.CONTINENT_NAME, Property.COUNTRY_NAME, Property.COUNTRY_ISO_CODE, Property.REGION_ISO_CODE,
Property.REGION_NAME, Property.CITY_NAME, Property.LOCATION
));
static final Set<Property> DEFAULT_COUNTRY_PROPERTIES = Collections.unmodifiableSet(EnumSet.of(
Property.CONTINENT_NAME, Property.COUNTRY_ISO_CODE
Property.CONTINENT_NAME, Property.COUNTRY_NAME, Property.COUNTRY_ISO_CODE
));
static final Set<Property> DEFAULT_ASN_PROPERTIES = Collections.unmodifiableSet(EnumSet.of(
Property.IP, Property.ASN, Property.ORGANIZATION_NAME, Property.NETWORK
Expand Down
15 changes: 10 additions & 5 deletions ...t-geoip/src/yamlRestTest/resources/rest-api-spec/test/ingest_geoip/20_geoip_processor.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,13 @@
index: test
id: 1
- match: { _source.field1: "128.101.101.101" }
- length: { _source.geoip: 6 }
- length: { _source.geoip: 7 }
- match: { _source.geoip.city_name: "Minneapolis" }
- match: { _source.geoip.country_iso_code: "US" }
- match: { _source.geoip.location.lon: -93.2548 }
- match: { _source.geoip.location.lat: 44.9399 }
- match: { _source.geoip.region_iso_code: "US-MN" }
- match: { _source.geoip.country_name: "United States" }
- match: { _source.geoip.region_name: "Minnesota" }
- match: { _source.geoip.continent_name: "North America" }

Expand Down Expand Up @@ -69,12 +70,13 @@
id: 1
- match: { _source.field1: ["128.101.101.101", "127.0.0.1"] }
- length: { _source.geoip: 2 }
- length: { _source.geoip.0: 6 }
- length: { _source.geoip.0: 7 }
- match: { _source.geoip.0.city_name: "Minneapolis" }
- match: { _source.geoip.0.country_iso_code: "US" }
- match: { _source.geoip.0.location.lon: -93.2548 }
- match: { _source.geoip.0.location.lat: 44.9399 }
- match: { _source.geoip.0.region_iso_code: "US-MN" }
- match: { _source.geoip.0.country_name: "United States" }
- match: { _source.geoip.0.region_name: "Minnesota" }
- match: { _source.geoip.0.continent_name: "North America" }
- match: { _source.geoip.1: null }
Expand Down Expand Up @@ -109,12 +111,13 @@
index: test
id: 1
- match: { _source.field1: ["127.0.0.1", "128.101.101.101", "128.101.101.101"] }
- length: { _source.geoip: 6 }
- length: { _source.geoip: 7 }
- match: { _source.geoip.city_name: "Minneapolis" }
- match: { _source.geoip.country_iso_code: "US" }
- match: { _source.geoip.location.lon: -93.2548 }
- match: { _source.geoip.location.lat: 44.9399 }
- match: { _source.geoip.region_iso_code: "US-MN" }
- match: { _source.geoip.country_name: "United States" }
- match: { _source.geoip.region_name: "Minnesota" }
- match: { _source.geoip.continent_name: "North America" }

Expand Down Expand Up @@ -196,8 +199,9 @@
index: test
id: 1
- match: { _source.field1: "128.101.101.101" }
- length: { _source.geoip: 2 }
- length: { _source.geoip: 3 }
- match: { _source.geoip.country_iso_code: "US" }
- match: { _source.geoip.country_name: "United States" }
- match: { _source.geoip.continent_name: "North America" }

---
Expand Down Expand Up @@ -259,12 +263,13 @@
index: test
id: 2
- match: { _source.field1: "128.101.101.101" }
- length: { _source.geoip: 6 }
- length: { _source.geoip: 7 }
- match: { _source.geoip.city_name: "Minneapolis" }
- match: { _source.geoip.country_iso_code: "US" }
- match: { _source.geoip.location.lon: -93.2548 }
- match: { _source.geoip.location.lat: 44.9399 }
- match: { _source.geoip.region_iso_code: "US-MN" }
- match: { _source.geoip.country_name: "United States" }
- match: { _source.geoip.region_name: "Minnesota" }
- match: { _source.geoip.continent_name: "North America" }

Expand Down