[8.x](backport #40301) Disable kubeadm for filebeat (both filbeat.input and filbeat.autodiscover)

[mergify]

• Enhancement

NOTE: Dont merge until elastic/elastic-agent-autodiscover#98 is also merged

Proposed commit message

WHAT: Provide a configuration option to disable /api/v1/namespaces/kube-system/configmaps/kubeadm-config for filebeat autodiscovery and add_kubernetes_metadata processor. In more details user will disable the api requests by using configuration use_kubeadm: false

In this PR we added the option in order to support both filbeat.input and filebeat.autodiscover blocks of configuration.

WHY: https://github.com/elastic/enhancements/issues/21172

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

1. Follow same steps as described [Kubernetes - Metricbeat] Providing config option to disable Kubeadm config api requests #40086
2. The filebeat configuration should contain only autodiscover config: https://github.com/elastic/beats/blob/main/deploy/kubernetes/filebeat-kubernetes.yaml#L133C1-L150C42

Related issues

• Relates #[Kubernetes - Metricbeat] Providing config option to disable Kubeadm config api requests #40086
• Relates #[META Issue]Enhancement for being able to toggle the kubeadm-config api calls  #40276

Screenshots

LOG processing continues successfully:

For the filebeat.input block:

LOG processing continues successfully:

---

This is an automatic backport of pull request #40301 done by [Mergify](https://mergify.com).

[mergify]

Cherry-pick of 89cd840 has failed:

On branch mergify/bp/8.x/pr-40301
Your branch is up to date with 'origin/8.x'.

You are currently cherry-picking commit 89cd840997.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   CHANGELOG.next.asciidoc
	modified:   dev-tools/kubernetes/filebeat/Dockerfile.debug
	modified:   dev-tools/kubernetes/heartbeat/Dockerfile.debug
	modified:   dev-tools/kubernetes/metricbeat/Dockerfile.debug
	modified:   filebeat/_meta/config/filebeat.autodiscover.reference.yml.tmpl
	modified:   filebeat/_meta/test/docs/01_playground/filebeat.yaml
	modified:   filebeat/filebeat.reference.yml
	modified:   libbeat/autodiscover/providers/kubernetes/config.go
	modified:   libbeat/autodiscover/providers/kubernetes/pod.go
	modified:   libbeat/autodiscover/providers/kubernetes/service.go
	modified:   libbeat/processors/add_kubernetes_metadata/config.go
	modified:   libbeat/processors/add_kubernetes_metadata/docs/add_kubernetes_metadata.asciidoc
	modified:   libbeat/processors/add_kubernetes_metadata/kubernetes.go
	modified:   metricbeat/docs/modules/kubernetes.asciidoc
	modified:   metricbeat/metricbeat.reference.yml
	modified:   metricbeat/module/kubernetes/_meta/config.reference.yml
	modified:   x-pack/filebeat/filebeat.reference.yml
	modified:   x-pack/metricbeat/metricbeat.reference.yml

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   metricbeat/module/kubernetes/util/kubernetes.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[elasticmachine]

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)