Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Agent K8S workload attestor doesn't pull container id from CGroups correctly #949

Closed
calaniz opened this issue Jun 6, 2019 · 2 comments
Closed

Comments

@calaniz
Copy link
Contributor

calaniz commented Jun 6, 2019

  • Version: 0.8.0
  • Platform: Linux weebly.local 3.10.0-862.14.4.el7.x86_64 CLI authentication #1 SMP Wed Sep 26 15:12:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
  • Subsystem: agent k8s workload attestor

The agent k8s workload attestor is logging

time="2019-06-06T20:21:41Z" level=warning msg="container id not found; giving up" attempt=1 container_id=docker-ab704ba978feea085662fd58c3448624de9287dd50c7d01608b27b08f96c8725.scope subsystem_name=builtin.k8s
time="2019-06-06T20:21:41Z" level=error msg="Failed to collect all selectors for PID 742: workload attestor \"k8s\" failed: rpc error: code = Unknown desc = k8s: no selectors found" subsystem_name=workload_api

It is incorrectly identifying the container id as docker-ab704ba978feea085662fd58c3448624de9287dd50c7d01608b27b08f96c8725.scope

Looking at /proc/29106/cgroup for example I see information in the format

11:blkio:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc68f332c_8897_11e9_9797_525400c9c704.slice/docker-ab704ba978feea085662fd58c3448624de9287dd50c7d01608b27b08f96c8725.scope

vs what getContainerIDFromCGroups is expecting

11:hugetlb:/kubepods/burstable/pod2c48913c-b29f-11e7-9350-020968147796/9bca8d63d5fa610783847915bcff0ecac1273e5b4bed3f6fa1b07350e0135961

I've added some additional logic to trim the prefix/suffix.

@evan2645
Copy link
Member

evan2645 commented Jun 6, 2019

Thanks for reporting this @calaniz. What version of kubernetes and docker are you running?

@calaniz
Copy link
Contributor Author

calaniz commented Jun 7, 2019

@evan2645

Kubernetes: v1.13.2
Docker: 1.13.1

Docker is configured to use the "systemd" cgroups driver.

Looking at k8s source, it is clear that these types of cgroup hierarchies were introduced sometime.
https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/cm/pod_container_manager_linux_test.go

note the "container.scope" cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants