[Fleet] Disallow some egress-specific inputs for agentless integrations #206074

criamico · 2025-01-09T14:21:34Z

Closes #202091

Summary

Disallow some egress-specific inputs for agentless integrations.

In the policy editor, when Setup technology dropdown is set to Agentless, hide the rendering of configuration for inputs that have type matching the blocklist and ensure that these inputs are set to enabled: false
- tcp, udp, winlog, http_endpoint, filestream should be disabled when supports_agentless: true
At the API level, throw an error if attempting to enable a disallowed input type

Testing

Simulate agentless env with following setup in kibana.dev.yml:

xpack.cloud.id: 'anything-to-pass-cloud-validation-checks'
xpack.fleet.agentless.enabled: true
xpack.fleet.agentless.api.url: 'https://localhost:8443'
xpack.fleet.agentless.api.tls.certificate: './config/certs/ess-client.crt'
xpack.fleet.agentless.api.tls.key: './config/certs/ess-client.key'
xpack.fleet.agentless.api.tls.ca: './config/certs/ca.crt'

-Apply this patch to prevent attempt to create agentless pod (the agentless policy creation fails without the patch)

Install the following test integration, that has a bunch of different inputs to simulate this specific case and is enabled for agentless (it shows the setup technology as well)
agentless_package_links-0.0.2.zip

curl -XPOST -H 'content-type: application/zip' -H 'kbn-xsrf: true' http://localhost:5601/YOUR_PATH/api/fleet/epm/packages -u elastic:changeme --data-binary @agentless_package_links-0.0.2.zip

Navigate to the integrations page, find the above integration and test that switching between agent-based/agentless the enabled inputs change as follows:

Verify that the preview flyout has the correct inputs based on the selected deployment mode

Verify that the api throws an error when attempting to enable any of the disallowed types

Checklist

Unit or functional tests were updated or added to match the most common scenarios
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

criamico · 2025-01-15T14:39:17Z

...ons/fleet/sections/agent_policy/create_package_policy_page/single_page_layout/hooks/form.tsx

@@ -291,6 +311,41 @@ export function useOnSubmit({
    }
  }, [packagePolicy, agentPolicies, updatePackagePolicy, canUseMultipleAgentPolicies]);

+  const { handleSetupTechnologyChange, selectedSetupTechnology } = useSetupTechnology({


I moved the useSetupTechnology inside the main hook, it was outside and was causing unexpected behaviours with state updates.

elasticmachine · 2025-01-15T14:39:50Z

Pinging @elastic/fleet (Team:Fleet)

criamico · 2025-01-15T16:36:33Z

@elasticmachine merge upstream

criamico · 2025-01-16T10:16:57Z

@elasticmachine merge upstream

criamico · 2025-01-16T13:48:04Z

@elasticmachine merge upstream

juliaElastic · 2025-01-16T15:49:33Z

The UI works fine, I am seeing something strange when trying to update an existing package policy with the API, changing networking_vars-udp input to enabled:true. The API request doesn't fail validation as expected.

PUT kbn:/api/fleet/package_policies/334fba75-a15d-495e-97d7-d4ac288555ae
{
  "package": {
    "name": "agentless_package_links",
    "version": "0.0.2"
  },
  "name": "agentless_package_links-3",
  "namespace": "",
  "description": "",
  "policy_ids": [
    "74baf76c-7847-4c5d-8444-ed9e69ab80af"
  ],
  "supports_agentless": true,
  "vars": {},
  "inputs": {
    "sample-connectors-py": {
      "enabled": true,
      "vars": {},
      "streams": {}
    },
    "sample-logfile": {
      "enabled": true,
      "streams": {}
    },
    "networking_vars-udp": {
      "enabled": true,
      "streams": {}
    },
    "networking_vars-tcp": {
      "enabled": false,
      "streams": {}
    },
    "networking_vars-logfile": {
      "enabled": true,
      "streams": {}
    },
    "networking_vars-http_endpoint": {
      "enabled": false,
      "streams": {}
    },
    "networking_vars-filestream": {
      "enabled": false,
      "streams": {}
    },
    "networking_vars-winlog": {
      "enabled": false,
      "streams": {}
    },
    "networking_vars-httpjson": {
      "enabled": true,
      "streams": {}
    }
  }
}

# response
{
  "item": {
    "id": "334fba75-a15d-495e-97d7-d4ac288555ae",
    "version": "WzEwNTcsMV0=",
    "name": "agentless_package_links-3",
    "namespace": "",
    "description": "",
    "package": {
      "name": "agentless_package_links",
      "title": "Agentless Package with Links",
      "version": "0.0.2"
    },
    "enabled": true,
    "policy_id": "74baf76c-7847-4c5d-8444-ed9e69ab80af",
    "policy_ids": [
      "74baf76c-7847-4c5d-8444-ed9e69ab80af"
    ],
    "inputs": [
      {
        "type": "connectors-py",
        "policy_template": "sample",
        "enabled": true,
        "streams": [],
        "vars": {
          "connector_name": {
            "type": "text"
          },
          "connector_id": {
            "type": "text"
          }
        },
        "compiled_input": {
          "service_type": "test_connector"
        }
      },
      {
        "type": "logfile",
        "policy_template": "sample",
        "enabled": true,
        "streams": []
      },
      {
        "type": "udp",
        "policy_template": "networking_vars",
        "enabled": false,
        "streams": []
      },
      {
        "type": "tcp",
        "policy_template": "networking_vars",
        "enabled": false,
        "streams": []
      },
      {
        "type": "logfile",
        "policy_template": "networking_vars",
        "enabled": true,
        "streams": []
      },
      {
        "type": "http_endpoint",
        "policy_template": "networking_vars",
        "enabled": false,
        "streams": []
      },
      {
        "type": "filestream",
        "policy_template": "networking_vars",
        "enabled": false,
        "streams": []
      },
      {
        "type": "winlog",
        "policy_template": "networking_vars",
        "enabled": false,
        "streams": []
      },
      {
        "type": "httpjson",
        "policy_template": "networking_vars",
        "enabled": true,
        "streams": []
      }
    ],
    "supports_agentless": true,
    "revision": 6,
    "created_at": "2025-01-16T15:41:32.772Z",
    "created_by": "system",
    "updated_at": "2025-01-16T15:46:23.003Z",
    "updated_by": "elastic"
  }
}

criamico · 2025-01-16T16:28:16Z

The UI works fine, I am seeing something strange when trying to update an existing package policy with the API, changing networking_vars-udp input to enabled:true. The API request doesn't fail validation as expected.

@juliaElastic I think that this happens only when using the "simplified" handler, as it uses defaults to create the policy. If you notice, it gives back the "expected" values, but as you said it doesn't really fail validation. I'll take a look on how to do it.
This doesn't happen if you try to update a policy using the format inputs: [].

criamico · 2025-01-21T09:37:48Z

@elasticmachine merge upstream

criamico · 2025-01-21T13:07:56Z

@elasticmachine merge upstream

criamico · 2025-01-22T09:46:12Z

@elasticmachine merge upstream

elasticmachine · 2025-01-22T10:56:32Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 5fbf147
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-206074-5fbf1471d1ee
Security Deployment

Failed CI Steps

Jest Tests #2

Test Failures

[job] [logs] Jest Tests #2 / RegisteredAttachmentsPropertyActions renders the correct number of actions

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`fleet`	1.7MB	1.7MB	+1.0KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`fleet`	172.4KB	172.5KB	+93.0B

History

cc @criamico

seanrathier · 2025-01-22T16:06:51Z

.../functional/test_suites/security/ftr/cloud_security_posture/agentless/cis_integration_gcp.ts

@@ -44,7 +44,7 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) {
      mockApiServer.close();
    });

-    describe('Agentless CIS_GCP Single Account Launch Cloud shell', () => {
+    describe.skip('Agentless CIS_GCP Single Account Launch Cloud shell', () => {


Follow up here https://github.com/elastic/security-team/issues/11629

Thanks Sean!

No problem, I tried running the FTRs by changing the version of the CSPM integration, but it still failed.

seanrathier · 2025-01-22T16:07:17Z

.../functional/test_suites/security/ftr/cloud_security_posture/agentless/cis_integration_gcp.ts

@@ -61,7 +61,7 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) {
      });
    });

-    describe('Agentless CIS_GCP ORG Account Launch Cloud Shell', () => {
+    describe.skip('Agentless CIS_GCP ORG Account Launch Cloud Shell', () => {


Follow up here https://github.com/elastic/security-team/issues/11629

kibanamachine · 2025-01-22T16:22:13Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12912546497

kibanamachine · 2025-01-22T16:27:30Z

💔 All backports failed

Status	Branch	Result
❌	8.x	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 206074

Questions ?

Please refer to the Backport tool documentation

criamico · 2025-01-23T09:32:09Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…grations (#206074) (#208004) # Backport This will backport the following commits from `main` to `8.x`: - [[Fleet] Disallow some egress-specific inputs for agentless integrations (#206074)](#206074)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

…ns (elastic#206074) Closes elastic#202091 ## Summary Disallow some egress-specific inputs for agentless integrations. - In the policy editor, when Setup technology dropdown is set to Agentless, hide the rendering of configuration for inputs that have type matching the blocklist and ensure that these inputs are set to `enabled: false` - `tcp, udp, winlog, http_endpoint, filestream` should be disabled when `supports_agentless: true` - At the API level, throw an error if attempting to enable a disallowed input type ### Testing Simulate agentless env with following setup in `kibana.dev.yml`: ``` xpack.cloud.id: 'anything-to-pass-cloud-validation-checks' xpack.fleet.agentless.enabled: true xpack.fleet.agentless.api.url: 'https://localhost:8443' xpack.fleet.agentless.api.tls.certificate: './config/certs/ess-client.crt' xpack.fleet.agentless.api.tls.key: './config/certs/ess-client.key' xpack.fleet.agentless.api.tls.ca: './config/certs/ca.crt' ``` -Apply [this patch](https://gist.github.com/jen-huang/dfc3e02ceb63976ad54bd1f50c524cb4) to prevent attempt to create agentless pod (the agentless policy creation fails without the patch) - Install the following test integration, that has a bunch of different inputs to simulate this specific case and is enabled for agentless (it shows the setup technology as well) [agentless_package_links-0.0.2.zip](https://github.com/user-attachments/files/18425895/agentless_package_links-0.0.2.zip) ``` curl -XPOST -H 'content-type: application/zip' -H 'kbn-xsrf: true' http://localhost:5601/YOUR_PATH/api/fleet/epm/packages -u elastic:changeme --data-binary @agentless_package_links-0.0.2.zip ``` - Navigate to the integrations page, find the above integration and test that switching between agent-based/agentless the enabled inputs change as follows: <img width="1288" alt="Screenshot 2025-01-15 at 15 30 28" src="https://github.com/user-attachments/assets/6abd45d7-1bd8-465a-af29-4c34940b32e3" /> <img width="1072" alt="Screenshot 2025-01-15 at 15 31 18" src="https://github.com/user-attachments/assets/6957562f-08a6-403a-8725-1a654e443537" /> - Verify that the preview flyout has the correct inputs based on the selected deployment mode <img width="863" alt="Screenshot 2025-01-15 at 15 32 19" src="https://github.com/user-attachments/assets/ceca1f5d-249c-4ee1-9295-6f01ae21fdb4" /> <img width="862" alt="Screenshot 2025-01-15 at 15 33 33" src="https://github.com/user-attachments/assets/f43562d7-633e-4f0a-bfc1-19e89aef7659" /> - Verify that the api throws an error when attempting to enable any of the disallowed types <img width="1774" alt="Screenshot 2025-01-15 at 15 36 03" src="https://github.com/user-attachments/assets/2b4d24a3-5adc-4ab2-bbad-83b44d348763" /> ### Checklist - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

amolnater-qasource · 2025-01-28T08:35:02Z

Hi Team,
While setting up environment for this on 8.18.0-SNAPSHOT, we are getting errors while adding package in agentless mode.

Self-Managed:

Cloud:

Agents.-.Fleet.-.Elastic.-.Google.Chrome.2025-01-27.17-45-06.mp4

Steps:

Added the xpack to the kibana.yml:

xpack.cloud.id: 'anything-to-pass-cloud-validation-checks'
xpack.fleet.agentless.enabled: true
xpack.fleet.agentless.api.url: 'https://localhost:8443'
xpack.fleet.agentless.api.tls.certificate: './config/certs/ess-client.crt'
xpack.fleet.agentless.api.tls.key: './config/certs/ess-client.key'
xpack.fleet.agentless.api.tls.ca: './config/certs/ca.crt'

Run the API to get the :
curl -XPOST -H 'content-type: application/zip' -H 'kbn-xsrf: true' http://localhost:5601/api/fleet/epm/packages -u elastic:changeme --data-binary @agentless_package_links-0.0.2.zip

Please let us know if we are missing anything here.

cc: @criamico

Thanks!!

## Summary Fixes a bug introduced with #206074 When a user selected a single account and switched the deployment method to `agentless`, it reverted to the organization selection. This PR removes[ the code](https://github.com/elastic/kibana/pull/206074/files#diff-a169e222e1116859e05821170ab592903c2943613dfe0035c32a4b9d42e5634fR330) that was causing the bug. https://github.com/user-attachments/assets/9007c35b-8454-4f7f-8811-bd39ba7ac99e ⚠️ **Important** ⚠️ One side effect of this change is that when we switch back from agentless to agent-based, the inputs that were previously hidden and disabled are now visible but they are disabled. This is not ideal but offers the user the option to re enable them, but at least doesn't try to forcefully switch them to initial state, that was the cause of the current bug. https://github.com/user-attachments/assets/520e361b-2789-4081-bb91-c75db88d5431 ### Checklist - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

## Summary Fixes a bug introduced with elastic#206074 When a user selected a single account and switched the deployment method to `agentless`, it reverted to the organization selection. This PR removes[ the code](https://github.com/elastic/kibana/pull/206074/files#diff-a169e222e1116859e05821170ab592903c2943613dfe0035c32a4b9d42e5634fR330) that was causing the bug. https://github.com/user-attachments/assets/9007c35b-8454-4f7f-8811-bd39ba7ac99e ⚠️ **Important** ⚠️ One side effect of this change is that when we switch back from agentless to agent-based, the inputs that were previously hidden and disabled are now visible but they are disabled. This is not ideal but offers the user the option to re enable them, but at least doesn't try to forcefully switch them to initial state, that was the cause of the current bug. https://github.com/user-attachments/assets/520e361b-2789-4081-bb91-c75db88d5431 ### Checklist - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> (cherry picked from commit e758f32)

# Backport This will backport the following commits from `main` to `8.x`: - [[Fleet] Fix uncorrect override of cspm inputs (#208562)](#208562)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Cristina Amico <criamico@users.noreply.github.com>

[Fleet] Disallow some egress-specific inputs for agentless integrations

a82bdf7

criamico self-assigned this Jan 9, 2025

criamico added Team:Fleet Team label for Observability Data Collection Fleet team release_note:skip Skip the PR/issue when compiling release notes labels Jan 9, 2025

criamico added 9 commits January 13, 2025 15:49

Move inputs change

94b7ed3

Move changes inside hook

91689af

Hide disallowed inputs

d0520c4

Fix types and remove unnecessary change

0309057

Enable validation in backend

9ff6982

Fix tests

dc92224

Update preview flyout

8efed2a

Fix state change in form

7ab056e

Merge branch 'main' into 202091_agentless_disable_specific_inputs

e3f5d7a

criamico added v9.0.0 backport:skip This commit does not require backporting labels Jan 15, 2025

criamico commented Jan 15, 2025

View reviewed changes

criamico marked this pull request as ready for review January 15, 2025 14:39

criamico requested a review from a team as a code owner January 15, 2025 14:39

remove comment

a4c48a2

criamico added 2 commits January 16, 2025 11:03

Merge main and fix conflicts

28c9faa

fix missing function

448772b

Merge branch 'main' into 202091_agentless_disable_specific_inputs

5c9a92e

Merge branch 'main' into 202091_agentless_disable_specific_inputs

bee8d7f

juliaElastic approved these changes Jan 17, 2025

View reviewed changes

criamico added the ci:project-deploy-security Create a Security Serverless Project label Jan 20, 2025

Skip failing test

74c838a

criamico requested a review from a team as a code owner January 20, 2025 14:57

criamico requested a review from seanrathier January 20, 2025 15:49

criamico added 2 commits January 20, 2025 17:23

Skip other failing test

f85465c

Skip other failing test

73842c5

Merge branch 'main' into 202091_agentless_disable_specific_inputs

217abbf

Merge branch 'main' into 202091_agentless_disable_specific_inputs

2517c4f

Merge branch 'main' into 202091_agentless_disable_specific_inputs

5fbf147

seanrathier approved these changes Jan 22, 2025

View reviewed changes

criamico added v8.18.0 backport:prev-minor Backport to (9.0) the previous minor version (i.e. one version back from main) and removed backport:skip This commit does not require backporting labels Jan 22, 2025

criamico merged commit 59e81ee into elastic:main Jan 22, 2025
13 checks passed

criamico deleted the 202091_agentless_disable_specific_inputs branch January 22, 2025 16:22

criamico mentioned this pull request Jan 23, 2025

[8.x] [Fleet] Disallow some egress-specific inputs for agentless integrations (#206074) #208004

Merged

criamico mentioned this pull request Jan 28, 2025

[Fleet] Fix uncorrect override of cspm inputs #208562

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Fleet] Disallow some egress-specific inputs for agentless integrations #206074

[Fleet] Disallow some egress-specific inputs for agentless integrations #206074

criamico commented Jan 9, 2025 •

edited by kibanamachine

Loading

criamico Jan 15, 2025

elasticmachine commented Jan 15, 2025

criamico commented Jan 15, 2025

criamico commented Jan 16, 2025

criamico commented Jan 16, 2025

juliaElastic commented Jan 16, 2025

criamico commented Jan 16, 2025 •

edited

Loading

criamico commented Jan 21, 2025

criamico commented Jan 21, 2025

criamico commented Jan 22, 2025

elasticmachine commented Jan 22, 2025 •

edited

Loading

seanrathier Jan 22, 2025

criamico Jan 22, 2025

seanrathier Jan 22, 2025 •

edited

Loading

seanrathier Jan 22, 2025

kibanamachine commented Jan 22, 2025

kibanamachine commented Jan 22, 2025

criamico commented Jan 23, 2025

amolnater-qasource commented Jan 28, 2025

[Fleet] Disallow some egress-specific inputs for agentless integrations #206074

[Fleet] Disallow some egress-specific inputs for agentless integrations #206074

Conversation

criamico commented Jan 9, 2025 • edited by kibanamachine Loading

Summary

Testing

Checklist

criamico Jan 15, 2025

Choose a reason for hiding this comment

elasticmachine commented Jan 15, 2025

criamico commented Jan 15, 2025

criamico commented Jan 16, 2025

criamico commented Jan 16, 2025

juliaElastic commented Jan 16, 2025

criamico commented Jan 16, 2025 • edited Loading

criamico commented Jan 21, 2025

criamico commented Jan 21, 2025

criamico commented Jan 22, 2025

elasticmachine commented Jan 22, 2025 • edited Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Async chunks

Page load bundle

History

seanrathier Jan 22, 2025

Choose a reason for hiding this comment

criamico Jan 22, 2025

Choose a reason for hiding this comment

seanrathier Jan 22, 2025 • edited Loading

Choose a reason for hiding this comment

seanrathier Jan 22, 2025

Choose a reason for hiding this comment

kibanamachine commented Jan 22, 2025

kibanamachine commented Jan 22, 2025

💔 All backports failed

Manual backport

Questions ?

criamico commented Jan 23, 2025

💚 All backports created successfully

Questions ?

amolnater-qasource commented Jan 28, 2025

criamico commented Jan 9, 2025 •

edited by kibanamachine

Loading

criamico commented Jan 16, 2025 •

edited

Loading

elasticmachine commented Jan 22, 2025 •

edited

Loading

seanrathier Jan 22, 2025 •

edited

Loading