Enable the Cloud Security Posture Kibana plugin #767

eyalkraft · 2022-03-30T14:31:49Z

Enable the Cloud Security Posture Kibana plugin by default when using elastic-package.

Relates Add Kubernetes CIS Benchmark integration integrations#2930

elasticmachine · 2022-03-30T14:42:56Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-04-05T14:32:18.738+0000
Duration: 26 min 14 sec

Test stats 🧪

Test	Results
Failed	0
Passed	658
Skipped	0
Total	658

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

eyalkraft · 2022-03-30T14:57:40Z

Maybe this fails because of dependency with this recent change in Elasticsearch?

add privileges for kIbana_system user to serve cloud security posture… elasticsearch#84941

I believe the Elasticsearch SNAPSHOT binaries don't include this change yet.

mtojek

Could you please fill in the issue description and link to the original issue?

Please take a look at the build artifacts. It looks like Kibana doesn't support it:

Attaching to elastic-package-stack_kibana_1
�[36mkibana_1                     |�[0m [2022-03-30T14:42:56.139+00:00][INFO ][plugins-service] Plugin "metricsEntities" is disabled.
�[36mkibana_1                     |�[0m [2022-03-30T14:42:56.207+00:00][FATAL][root] Error: Unknown configuration key(s): "xpack.cloudSecurityPosture.enabled". Check for spelling errors and ensure that expected plugins are installed.
�[36mkibana_1                     |�[0m     at ensureValidConfiguration (/usr/share/kibana/src/core/server/config/ensure_valid_configuration.js:35:11)
�[36mkibana_1                     |�[0m     at Server.preboot (/usr/share/kibana/src/core/server/server.js:164:5)
�[36mkibana_1                     |�[0m     at Root.preboot (/usr/share/kibana/src/core/server/root/index.js:48:14)
�[36mkibana_1                     |�[0m     at bootstrap (/usr/share/kibana/src/core/server/bootstrap.js:99:9)
�[36mkibana_1                     |�[0m     at Command.<anonymous> (/usr/share/kibana/src/cli/serve/serve.js:216:5)
�[36mkibana_1                     |�[0m 
�[36mkibana_1                     |�[0m  FATAL  Error: Unknown configuration key(s): "xpack.cloudSecurityPosture.enabled". Check for spelling errors and ensure that expected plugins are installed.
�[36mkibana_1                     |�[0m

Is it generally available in 8.x?

eyalkraft · 2022-03-30T16:03:29Z

Hey, It's not GA yet (but merged to kibana/main).
We plan on releasing it as beta with 8.3

Kibana 8.2-SNAPSHOT should contain the plugin I believe

mtojek · 2022-03-30T18:49:29Z

Kibana 8.2-SNAPSHOT should contain the plugin I believe

I can confirm that 8.2 stack booted up without issues.

That complicates things as we need to support another Kibana config variant. I guess we can help with this one. Sadly Kibana can't ignore unknown configuration properties.

Please tell me how urgent is this change as we have other stuff on the board?

eyalkraft · 2022-03-31T07:10:33Z

Is this a blocker for merging the integration?
If so, it is urgent for us. We can open the PR to support the additional variant if you are short in time - so you will only need to review.

If this is not a blocker for merging the integration, We can work with elastic-package just fine configuring the kibana.yml in our own local profiles.
This flag is only required for installing our integration.

mtojek · 2022-03-31T07:44:53Z

Then it's a blocker as correct installation of the integration is required as part of the validation process. If you have spare time to work on this, then please assign us as reviewers.

AFAIR here are the required steps:

Prepare an additional config file based on kibana_config_8x.yml, for example kibana_config_80.yml.
Register the file in here.
Add new entries to managedProfileFiles.
This map keeps tracking of all configs and it's used to create files in ~/.elastic-package/profiles. Probably you will have to add virtual records for any non-existing 8.0 configs too. We don't want to duplicate config files for Elasticsearch and Elastic Agent in the source code. You can find all config variants here by searching for STACK_VERSION_VARIANT.

As we can see, the CI failed already for many steps but succeeded for 8x, as of now it's 8.2.0-SNAPSHOT, which means that it can boot up correctly with the plugin. We just need to strip the entry from the other Kibana config.

eyalkraft · 2022-04-04T08:59:32Z

/test

This reverts commit 1faa0bd.

eyalkraft · 2022-04-04T13:44:34Z

Added internal/profile/_static/kibana_config_80.yml file for stack versions 8.0 to 8.1
Registered the new file (and virtual files for elasticsearch and elastic-agent) in profile/static.go
Added the new entries to managedProfileFiles in profile/profile.go
Modified stack/variants.go to support the additional variant
Added stack/variants_test.go to validate and clarify the behavior of stack/variants.go

mtojek

Left a small comment around selectStackVersion. Otherwise, it's ready for shipping. Well done!

mtojek · 2022-04-04T15:19:30Z

internal/stack/variants.go

 	if strings.HasPrefix(version, "8.") {
+		if len(version) > 2 && (int(version[2])-'0') < 2 {
+			return "80"
+		}
 		return "8x"
 	}


I guess that you can also parse the version as a semver version (the library is already used in the project) and check the major and minor. It might be more to catch :)

Nice!
I wasn't too comfortable with this as well.

mtojek · 2022-04-05T08:52:13Z

internal/stack/variants.go

@@ -6,20 +6,34 @@ package stack

 import (
 	"fmt"
-	"strings"
+
+	"github.com/Masterminds/semver/v3"


just a small comment, I see you imported the "v3". Is there anything missing in the github.com/Masterminds/semver as we are using the base release? If not, would you mind switching to that version?

Done.
I had some trouble finding the correct v1 syntax for the range constraint as it differs from the v3 a bit.

mtojek · 2022-04-05T08:53:17Z

internal/stack/variants.go

-	if strings.HasPrefix(version, "8.") {
-		return "8x"
+	if v, err := semver.NewVersion(version); err == nil {
+		if checkVersion(v, "8.0-0 - 8.1-0") {


👍

much clearer now!

mtojek · 2022-04-05T08:59:27Z

internal/stack/variants.go

 func stackVariantAsEnv(version string) string {
 	return fmt.Sprintf("STACK_VERSION_VARIANT=%s", selectStackVersion(version))
 }

 func selectStackVersion(version string) string {
-	if strings.HasPrefix(version, "8.") {
-		return "8x"
+	if v, err := semver.NewVersion(version); err == nil {


Actually, you can convert these if-conditions into a configurationVariantMap (key: constraint, value: config variant).

eyalkraft added 2 commits March 30, 2022 17:21

add to kibana.yml

7473d7a

remove newline

896be85

eyalkraft mentioned this pull request Mar 30, 2022

Add Kubernetes CIS Benchmark integration elastic/integrations#2930

Merged

4 tasks

mtojek requested a review from a team March 30, 2022 15:01

mtojek reviewed Mar 30, 2022

View reviewed changes

mtojek marked this pull request as draft March 31, 2022 07:44

eyalkraft self-assigned this Mar 31, 2022

Merge branch 'main' into enable-cloud-securty-posture-kibana-plugin

da82e1f

eyalkraft added 5 commits April 4, 2022 14:25

add 80 config file

d4ec8bf

fix license

d8f79cc

use symlinks

1faa0bd

Revert "use symlinks"

f3dc19d

This reverts commit 1faa0bd.

virtual files

569c710

eyalkraft marked this pull request as ready for review April 4, 2022 13:36

eyalkraft requested a review from mtojek April 4, 2022 13:44

mtojek requested a review from a team April 4, 2022 15:15

mtojek reviewed Apr 4, 2022

View reviewed changes

eyalkraft added 3 commits April 4, 2022 22:16

use semver

1153a98

fix static

4d2c446

retrigger stuck CI

1479d39

eyalkraft requested a review from mtojek April 5, 2022 07:01

mtojek reviewed Apr 5, 2022

View reviewed changes

eyalkraft added 3 commits April 5, 2022 13:51

add configuration variant map

c44413f

use old semver

898bd70

fix static check

cb91746

eyalkraft requested a review from mtojek April 5, 2022 14:34

mtojek approved these changes Apr 5, 2022

View reviewed changes

mtojek merged commit 6a600dc into elastic:main Apr 5, 2022

eyalkraft mentioned this pull request Jun 28, 2022

[Cloud Posture] Enable the cloud security posture plugin by default elastic/kibana#134417

Closed

4 tasks

ari-aviran mentioned this pull request Jul 27, 2022

Remove or deprecate xpack.cloudSecurityPosture.enabled elastic/kibana#137262

Open

3 tasks

kfirpeled added the Team:Cloud Security Posture label Nov 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable the Cloud Security Posture Kibana plugin #767

Enable the Cloud Security Posture Kibana plugin #767

eyalkraft commented Mar 30, 2022 •

edited

Loading

elasticmachine commented Mar 30, 2022 •

edited

Loading

Build stats

Test stats 🧪

eyalkraft commented Mar 30, 2022

mtojek left a comment

eyalkraft commented Mar 30, 2022

mtojek commented Mar 30, 2022

eyalkraft commented Mar 31, 2022

mtojek commented Mar 31, 2022

eyalkraft commented Apr 4, 2022

eyalkraft commented Apr 4, 2022

mtojek left a comment

mtojek Apr 4, 2022

eyalkraft Apr 4, 2022

mtojek Apr 5, 2022

eyalkraft Apr 5, 2022

mtojek Apr 5, 2022

mtojek Apr 5, 2022

eyalkraft Apr 5, 2022

Enable the Cloud Security Posture Kibana plugin #767

Enable the Cloud Security Posture Kibana plugin #767

Conversation

eyalkraft commented Mar 30, 2022 • edited Loading

elasticmachine commented Mar 30, 2022 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

eyalkraft commented Mar 30, 2022

mtojek left a comment

Choose a reason for hiding this comment

eyalkraft commented Mar 30, 2022

mtojek commented Mar 30, 2022

eyalkraft commented Mar 31, 2022

mtojek commented Mar 31, 2022

eyalkraft commented Apr 4, 2022

eyalkraft commented Apr 4, 2022

mtojek left a comment

Choose a reason for hiding this comment

mtojek Apr 4, 2022

Choose a reason for hiding this comment

eyalkraft Apr 4, 2022

Choose a reason for hiding this comment

mtojek Apr 5, 2022

Choose a reason for hiding this comment

eyalkraft Apr 5, 2022

Choose a reason for hiding this comment

mtojek Apr 5, 2022

Choose a reason for hiding this comment

mtojek Apr 5, 2022

Choose a reason for hiding this comment

eyalkraft Apr 5, 2022

Choose a reason for hiding this comment

eyalkraft commented Mar 30, 2022 •

edited

Loading

elasticmachine commented Mar 30, 2022 •

edited

Loading