-
Notifications
You must be signed in to change notification settings - Fork 75
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial definition of content pack[age]s #777
Conversation
6114b36
to
6900576
Compare
6900576
to
c6687d6
Compare
spec/content/manifest.spec.yml
Outdated
type: object | ||
additionalProperties: false | ||
definitions: | ||
interested_on: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems like a tricky thing to name. What do you think about discovery_fields
instead, in the interest of being more explicit about the behavior associated with this property?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I am not super happy with the name, so thanks for the suggestion!
I wouldn't include fields
at this level because maybe we can support other things, like other packages, but discovery
could fit.
discovery:
fields:
- name: "process.pid"
discovery:
packages:
- name: "system"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like leaving room for driving recommendations for other packages vs fields - I think this approach is solid.
@@ -23,25 +23,33 @@ type Filter struct { | |||
type FilterResult struct { | |||
Processed error | |||
Removed error | |||
|
|||
UnusedProcessors []Processor |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added detection of unused rules, used by now only on tests, we could warn in the future about this.
Opening for review so we can start testing it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Added some questions about the limits for this new type of package
spec/content/spec.yml
Outdated
totalSizeLimit: 50MB | ||
sizeLimit: 30MB |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are these limits related to the whole package ? As these are going to be mainly assets, probably we could set the same limits as the integration
packages. WDYT ?
totalSizeLimit: 250MB
sizeLimit: 150MB
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, in any case we will revisit this, as content packages are intended to support bigger sizes than other packages.
💚 Build Succeeded
History
cc @jsoriano |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
## Summary Related to #192484. This PR adding [new content package types and schemas](elastic/package-spec#777) so that content packages can be returned correctly from EPR to unblock development of those packages. The only current content package is `kubernetes_otel`. You will need to bump up the max allowed spec version and search with beta (prerelease) packages enabled to find it: ``` xpack.fleet.internal.registry.spec.max: '3.4' ``` Tests will come with the rest of work for #192484 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary Related to elastic#192484. This PR adding [new content package types and schemas](elastic/package-spec#777) so that content packages can be returned correctly from EPR to unblock development of those packages. The only current content package is `kubernetes_otel`. You will need to bump up the max allowed spec version and search with beta (prerelease) packages enabled to find it: ``` xpack.fleet.internal.registry.spec.max: '3.4' ``` Tests will come with the rest of work for elastic#192484 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit bd6533f) # Conflicts: # oas_docs/bundle.json # oas_docs/bundle.serverless.json # oas_docs/output/kibana.serverless.staging.yaml # oas_docs/output/kibana.serverless.yaml # oas_docs/output/kibana.staging.yaml # oas_docs/output/kibana.yaml # x-pack/plugins/fleet/server/types/rest_spec/epm.ts
…195690) # Backport This will backport the following commits from `main` to `8.x`: - [[UII] Add types to return content packages correctly (#195505)](#195505) Note that this includes manual changes to openapi spec files. This change is not present on the original PR as the main branch no longer uses openapi spec files. <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Jen Huang","email":"its.jenetic@gmail.com"},"sourceCommit":{"committedDate":"2024-10-09T20:08:18Z","message":"[UII] Add types to return content packages correctly (#195505)\n\n## Summary\r\n\r\nRelated to #192484. This PR adding [new content package types and\r\nschemas](elastic/package-spec#777) so that\r\ncontent packages can be returned correctly from EPR to unblock\r\ndevelopment of those packages.\r\n\r\nThe only current content package is `kubernetes_otel`. You will need to\r\nbump up the max allowed spec version and search with beta (prerelease)\r\npackages enabled to find it:\r\n```\r\nxpack.fleet.internal.registry.spec.max: '3.4'\r\n```\r\n\r\nTests will come with the rest of work for #192484\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"bd6533f30b58fc831670d400f25a61321379902c","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Fleet","v9.0.0","backport:prev-minor"],"number":195505,"url":"https://github.com/elastic/kibana/pull/195505","mergeCommit":{"message":"[UII] Add types to return content packages correctly (#195505)\n\n## Summary\r\n\r\nRelated to #192484. This PR adding [new content package types and\r\nschemas](elastic/package-spec#777) so that\r\ncontent packages can be returned correctly from EPR to unblock\r\ndevelopment of those packages.\r\n\r\nThe only current content package is `kubernetes_otel`. You will need to\r\nbump up the max allowed spec version and search with beta (prerelease)\r\npackages enabled to find it:\r\n```\r\nxpack.fleet.internal.registry.spec.max: '3.4'\r\n```\r\n\r\nTests will come with the rest of work for #192484\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"bd6533f30b58fc831670d400f25a61321379902c"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/195505","number":195505,"mergeCommit":{"message":"[UII] Add types to return content packages correctly (#195505)\n\n## Summary\r\n\r\nRelated to #192484. This PR adding [new content package types and\r\nschemas](elastic/package-spec#777) so that\r\ncontent packages can be returned correctly from EPR to unblock\r\ndevelopment of those packages.\r\n\r\nThe only current content package is `kubernetes_otel`. You will need to\r\nbump up the max allowed spec version and search with beta (prerelease)\r\npackages enabled to find it:\r\n```\r\nxpack.fleet.internal.registry.spec.max: '3.4'\r\n```\r\n\r\nTests will come with the rest of work for #192484\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"bd6533f30b58fc831670d400f25a61321379902c"}}]}] BACKPORT--> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
What does this PR do?
Add the initial definition for content packages.
On this initial version, they only support the distribution of kibana assets, like dashboards.
This package type introduces a discovery mechanism based on the data the package can work with. For example the following definition would indicate that the package provides resources that are useful when used with data that includes the field
process.pid
:The package registry could add support to look for packages that have support for certain fields. For example a request like the following one could be used to discover packages that include resources that can leverage the
process.pid
or theuser.id
fields.Why is it important?
There are some use cases where developers want to distribute resources that can be useful with some data, without coupling them to the mechanism used to collect this data.
These resources could be for example dashboards that can be used with data collected with other integrations or even with other agents as the OTEL collector.
Paves the foundations for the use cases described in #351.
Checklist
test/packages
that prove my change is effective.spec/changelog.yml
.Related issues