Adding tags using http plugin in json format

[xAnder1402]

Relevant telegraf.conf:

[[outputs.influxdb]]
  urls = ["http://influxdb:8086"]
  database = "telegraf"
  [outputs.influxdb.tagdrop]
    influxdb_tag = ["*"]
[[outputs.influxdb]]
  urls = ["http://influxdb:8086"]
  database = "weather"
  [outputs.influxdb.tagpass]
    influxdb_tag = ["weather"]
[[inputs.http]]
  urls = [
    "https://samples.openweathermap.org/data/2.5/weather?id=2172797&appid=b6907d289e10d714a6e88b30761fae22"
  ]
  data_format = "json"
  tag_keys = [
    "coord_lon", "coord_lat", "sys_country", "id", "name"
  ]
  json_string_fields = [
    "weather_main", "weather_description"
  ]
  json_name_key = "weather"
  json_time_key = "dt"
  json_time_format = "unix"
  [inputs.http.tags]
    influxdb_tag = "weather"

System info:

Influxdb version: 1.7.3
Telegraf version: 1.9.2
OS: CentOS Linux release 7.6.1810 (Core)
Docker: 18.09.1
Docker image: telegraf:alpine

Actual behavior:

Fields that are defined as tags (tag_keys) and as timestamp (json_time_key) are also stored as empty fields.
Fields that are defined as strings (json_string_fields) are not saved.
The parameter "json_name_key" does not change the measurement name.
Measurement name is "http".
Timestamp by parameter "json_time_key" added correctly.

Additional info:

SHOW FIELD KEYS ON weather FROM http
name: http

fieldKey	fieldType
clouds_all	float
cod	float
coord_lat	float
coord_lon	float
dt	float
id	float
main_grnd_level	float
main_humidity	float
main_pressure	float
main_sea_level	float
main_temp	float
main_temp_max	float
main_temp_min	float
sys_message	float
sys_sunrise	float
sys_sunset	float
weather_0_id	float
wind_deg	float
wind_speed	float

SHOW TAG KEYS ON weather FROM http
name: http

tagKey
coord_lat
coord_lon
host
id
influxdb_tag
name
sys_country
url

SELECT coord_lat, coord_lon, dt, id FROM http
Your query is syntactically correct but returned no results

[danielnelson]

Can you run telegraf --input-filter=http --test and also attach the output of the URL?

[xAnder1402]

In my case, I replaced some information, I marked it with bold.

telegraf --input-filter=http --test`
2019-01-23T10:24:11Z I! Starting Telegraf 1.9.2
2019-01-23T10:24:11Z I! Using config file: /etc/telegraf/telegraf.conf
http,coord_lat=**10.20**,coord_lon=**30.40**,host=**HOST**,id=**123456**,influxdb_tag=weather,name=**City**,sys_country=**CO**,url=https://api.openweathermap.org/data/2.5/weather?id\=**123456**&units\=metric&lang\=ru&appid\=**id_here** clouds_all=32,cod=200,main_grnd_level=1001.77,main_humidity=82,main_pressure=1001.77,main_sea_level=1026.78,main_temp=-9.73,main_temp_max=-9.73,main_temp_min=-9.73,sys_message=0.0031,sys_sunrise=1548223978,sys_sunset=1548254599,weather_0_id=802,wind_deg=89.5007,wind_speed=4.46 1548252419000000000

{
  "coord":{
    "lon":**10.20**,
    "lat":**30.40**
  },
  "weather":[
    {
      "id":802,
      "main":"Clouds",
      "description":"слегка облачно",
      "icon":"03d"
    }
  ],
  "base":"stations",
  "main":{
    "temp":-9.73,
    "pressure":1001.77,
    "humidity":82,
    "temp_min":-9.73,
    "temp_max":-9.73,
    "sea_level":1026.78,
    "grnd_level":1001.77
  },
  "wind":{
    "speed":4.46,
    "deg":89.5007
  },
  "clouds":{
    "all":32
  },
  "dt":1548252419,
  "sys":{
    "message":0.0031,
    "country":"**CO**",
    "sunrise":1548223978,
    "sunset":1548254599
  },
  "id":**123456**,
  "name":"**City**",
  "cod":200
}

[danielnelson]

I notice in the test output that the tags (coord_lat, coord_lon, etc) are not being reported as fields, perhaps they were once inserted as fields? Once a field has been created the show field keys command will always show them.

The json_name_key should point to a string key in the JSON document, for example it could be set to name and then the measurement name would be the city. If you just want the measurement name to be weather, which would make more sense to me, you can use name_override which is a setting you can use on any plugin.

[xAnder1402]

I notice in the test output that the tags (coord_lat, coord_lon, etc) are not being reported as fields, perhaps they were once inserted as fields? Once a field has been created the show field keys command will always show them.

Probably, but I used DROP MEASUREMENT http

Ok, I will use one name_override, but fields that are defined as strings (json_string_fields) are not saved.

[danielnelson]

I have seen some situations where the metric is not fully remove after running drop measurement, though I'm not sure the conditions that cause it to happen: influxdata/influxdb#9694

The name and description are a bit tricky, because it looks like there could be multiple items, but you could specify them like this:

  json_string_fields = [
    "weather_0_main", "weather_0_description"
  ]

BTW, you might want to keep an eye on this pull request too: #5125

[xAnder1402]

json_string_fields = [
    "weather_0_main", "weather_0_description"
]

Thanks, it's works.

This solves the empty field problem:
influx_inspect buildtsi -datadir /var/lib/influxdb/data/ -waldir /var/lib/influxdb/wal/