Skip to content

Commit

Permalink
[Telemetry] Full schema definition (#90273)
Browse files Browse the repository at this point in the history
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
  • Loading branch information
afharo and kibanamachine authored Mar 1, 2021
1 parent 4ea10d9 commit f44916b
Show file tree
Hide file tree
Showing 23 changed files with 988 additions and 188 deletions.
9 changes: 0 additions & 9 deletions .telemetryrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,6 @@
{
"output": "src/plugins/telemetry/schema/oss_plugins.json",
"root": "src/plugins/",
"exclude": [
"src/plugins/kibana_react/",
"src/plugins/testbed/",
"src/plugins/kibana_utils/"
]
},
{
"output": "src/plugins/telemetry/schema/legacy_plugins.json",
"root": "src/legacy/server/",
"exclude": []
}
]
17 changes: 17 additions & 0 deletions src/plugins/telemetry/schema/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Telemetry Schemas

This list of `.json` files describes the format of the payloads sent to the Remote Telemetry Service. All the files should follow the schema convention as defined in the `usage_collection` plugin and `@kbn/telemetry-tools`, with the addition of the type `pass_through`. This additional `type` indicates Kibana sends the payload as-is from the output of an external ES query.

There are currently 2 files:

- `oss_root.json`: Defines the schema for the payload from the root keys.
Manually maintained for now because the frequency it changes should be pretty low.
- `oss_plugins.json`: The schema for the content that will be nested in `stack_stats.kibana.plugins`.
It is automatically generated by `@kbn/telemetry-tools` based on the `schema` property provided by all the registered Usage Collectors via the `usageCollection.makeUsageCollector` API.
More details in the [Schema field](../../usage_collection/README.md#schema-field) chapter in the UsageCollection's docs.

NOTE: Despite its similarities to ES mappings, the intention of these files is not to define any index mappings. They should be considered as a tool to understand the format of the payload that will be sent when reporting telemetry to the Remote Service.

## Testing

Functional tests are defined at `test/api_integration/apis/telemetry/telemetry_local.ts`. They merge both files, and validates the actual output of the telemetry endpoint against the final schema.
3 changes: 0 additions & 3 deletions src/plugins/telemetry/schema/legacy_plugins.json

This file was deleted.

199 changes: 199 additions & 0 deletions src/plugins/telemetry/schema/oss_root.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,199 @@
{
"properties": {
"timestamp": {
"type": "date"
},
"cluster_uuid": {
"type": "keyword"
},
"cluster_name": {
"type": "keyword"
},
"version": {
"type": "keyword"
},
"collection": {
"type": "keyword"
},
"collectionSource": {
"type": "keyword"
},
"stack_stats": {
"properties": {
"data": {
"type": "array",
"items": {
"properties": {
"data_stream": {
"properties": {
"dataset": {
"type": "keyword"
},
"type": {
"type": "keyword"
}
}
},
"package": {
"properties": {
"name": {
"type": "keyword"
}
}
},
"shipper": {
"type": "keyword"
},
"pattern_name": {
"type": "keyword"
},
"index_count": {
"type": "long"
},
"ecs_index_count": {
"type": "long"
},
"doc_count": {
"type": "long"
},
"size_in_bytes": {
"type": "long"
}
}
}
},
"kibana": {
"properties": {
"timelion_sheet": {
"properties": {
"total": {
"type": "long"
}
}
},
"visualization": {
"properties": {
"total": {
"type": "long"
}
}
},
"search": {
"properties": {
"total": {
"type": "long"
}
}
},
"index_pattern": {
"properties": {
"total": {
"type": "long"
}
}
},
"dashboard": {
"properties": {
"total": {
"type": "long"
}
}
},
"graph_workspace": {
"properties": {
"total": {
"type": "long"
}
}
},
"count": {
"type": "short"
},
"indices": {
"type": "short"
},
"os": {
"properties": {
"platforms": {
"type": "array",
"items": {
"properties": {
"platform": {
"type": "keyword"
},
"count": {
"type": "short"
}
}
}
},
"platformReleases": {
"type": "array",
"items": {
"properties": {
"platformRelease": {
"type": "keyword"
},
"count": {
"type": "short"
}
}
}
},
"distros": {
"type": "array",
"items": {
"properties": {
"distro": {
"type": "keyword"
},
"count": {
"type": "short"
}
}
}
},
"distroReleases": {
"type": "array",
"items": {
"properties": {
"distroRelease": {
"type": "keyword"
},
"count": {
"type": "short"
}
}
}
}
}
},
"versions": {
"type": "array",
"items": {
"properties": {
"version": {
"type": "keyword"
},
"count": {
"type": "short"
}
}
}
},
"plugins": {
"properties": {
"THIS_WILL_BE_REPLACED_BY_THE_PLUGINS_JSON": {
"type": "text"
}
}
}
}
}
}
},
"cluster_stats": {
"type": "pass_through"
}
}
}
Loading

0 comments on commit f44916b

Please sign in to comment.