Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

processor.add_kubernetes_metadata.matchers.resource_type setting won't work #27476

Closed
supersxlx opened this issue Aug 18, 2021 · 6 comments
Closed
Labels
Team:Integrations Label for the Integrations team

Comments

@supersxlx
Copy link

Hi:
I started a filebeat on my kubernetes master node. trying to add "pod_name" to index;but filebeat reported “Error extracting container id - source value does not contain matcher's logs_path '/var/lib/kubelet/pods/'”
My filebeat config:

 `processors:
  - add_host_metadata:
        when.not.contains.tags: forwarded
  - add_cloud_metadata: null
  - add_docker_metadata: null
  - add_kubernetes_metadata:
        in_cluster: false 
        host: ${NODE_IP}
        kube_config: /root/.kube/config
        default_indexers.enabled: false
        default_matchers.enabled: false
         indexers: 
            - pod_uid:
         matchers:
             - logs_path:
                   logs_path: '/var/lib/kubelet/pods/'
                   resource_type: 'pod'
`  

I looked "matchers.go" , if my setting was right, then it will use "pod_id" do the match. but it's not .
So , what did i do wrong ? My input type is "container" , is this one get wrong?

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 18, 2021
@supersxlx
Copy link
Author

i try change type to "log", not helping.

@ruflin ruflin added the Team:Integrations Label for the Integrations team label Aug 19, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 19, 2021
@ChrsMark
Copy link
Member

Hi!

Your configuration looks quite similar to the example at https://www.elastic.co/guide/en/beats/filebeat/current/add-kubernetes-metadata.html#_logs_path. Could you provide the version of Filebeat you are using?

@jsoriano does this ring any bells? Could it be a regression?

@jsoriano
Copy link
Member

@jsoriano does this ring any bells? Could it be a regression?

No idea, I think that there haven't been significative changes there recently.

We would need to see the input configuration, and know on what paths the logs are. The configuration in add_kubernetes_metadata to use will depend on that.

@supersxlx could you please share your input configuration and an event that is not being correctly enriched with k8s metadata?

@ChrsMark
Copy link
Member

@tetianakravchenko do you think it could be solved by your changes at #28868? If so, we can close this one :)

@tetianakravchenko
Copy link
Contributor

@ChrsMark yes, I think it should be fixed in #28868

@supersxlx if you are trying to read logs from stdout, with the PR above config should look like:

processors:
- add_kubernetes_metadata:
   ....
    matchers:
    - logs_path:
         logs_path: '/var/log/pods/'
         resource_type: 'pod'

note that input path must be a subdirectory of /var/log/pods/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Integrations Label for the Integrations team
Projects
None yet
Development

No branches or pull requests

6 participants