CRI-O container support #1310

gnosek · 2019-02-05T19:18:06Z

CRI-O uses CRI so most of the work is done already but it differs in some important ways from containerd and has to be supported explicitly.

speedyguy17

looks mostly okay, some concerns about the string matching function though

speedyguy17 · 2019-02-11T17:57:04Z

userspace/libsinsp/runc.h

+namespace libsinsp {
+namespace runc {
+
+/// runc-based runtimes (Docker, containerd, CRI-O, probably others) use the same two cgroup layouts


i appreciate the comment, but still slightly confused. maybe just an exapmle for each of the runc runtimes we support?

Do you still find it confusing after looking at the instances of this struct (container_engine/cri.cpp, container_engine/docker_linux.cpp)?

less so, but one don't want to have to dig into the implementation to understand the api.

speedyguy17 · 2019-02-11T18:03:36Z

userspace/libsinsp/runc.cpp

+namespace {
+
+const size_t CONTAINER_ID_LENGTH = 64;
+const size_t REPORTED_CONTAINER_ID_LENGTH = 12;


maybe a note what these values represent?

I'm not sure what to add that wouldn't duplicate the constant names or the static_assert below.

what does "reported" mean here? is it what's reported to us? what we're reporting to someone else?

speedyguy17 · 2019-02-11T18:04:52Z

userspace/libsinsp/runc.cpp

+
+static_assert(REPORTED_CONTAINER_ID_LENGTH <= CONTAINER_ID_LENGTH, "Reported container ID length cannot be longer than actual length");
+
+bool detect_one_container_id(const std::string& cgroup, const std::string& prefix, const std::string& suffix, std::string& container_id)


should these two functions be static?

also quick comment on which are inputs and ouputs of this function

They're already in a private namespace.

I'll add a comment.

must have missed that they're still in the NS. that's fine then.

speedyguy17 · 2019-02-11T18:06:54Z

userspace/libsinsp/runc.h

+	const char* suffix;
+};
+
+/// If any of the cgroups of the thread in `tinfo` matches the `layout`, set `container_id` to the found id


what if multiple cgroups match the layout?

I assume we're assuming that impossible, and if that's the case ,we should perhaps note the assumption

Then we take the first matching one (AFAICT, the iteration order matches contents of /proc//cgroups). If the cgroups are inconsistent (matching different containers), there's no single right answer we can choose. What do you propose?

so long as we're okay with that, it's fine, but we should at least be explicit in that "if it matches multiple, we'll only return the first" or something

speedyguy17 · 2019-02-11T18:09:44Z

userspace/libsinsp/runc.cpp

+
+bool detect_container_id(const std::string& cgroup, const libsinsp::runc::cgroup_layout *layout, std::string &container_id)
+{
+	for(size_t i = 0; layout[i].prefix && layout[i].suffix; ++i)


this way of doing bounds checking is somewhat risky, as it depends on someone putting null pointers at the end of arrays put in.

We should be able to statically determine the size of those arrays with sizeof(X)/sizeof(X[0]) and then that value can be passed in, or even better, we can wrap the array in a struct which stores the size of the array, calculated at compile time

sizeof won't work across function calls. Given that we have only two of these arrays, hardcoded in the source and not modifiable at runtime, I'm not sure it's a big deal (if you insist, I'd probably go for std::array).

i don't think the number of instance is the issue as much as the fact that depending on the last entry of an array to be null in order to prevent a segfault is not really idiomatic, and there's no compelling reason to do it that way when safer options exist.

I didn't mean computing sizeof in the function itself, but along with the array
static const int foo = {1,5,7,2,7};
static const foo_length = sizeof(foo) / sizeof(foo[0]);

Then you use foo_length as your loop bounds, as it's clear and safe.

std::array is fine or even vector would work fine here as well

userspace/libsinsp/runc.cpp

speedyguy17 · 2019-02-11T18:33:38Z

userspace/libsinsp/runc.cpp

+
+	static_assert(std::string::npos == (size_t)(-1), "std::string::npos must be largest valid size_t");
+	size_t invalid_ch_pos = cgroup.find_first_not_of(CONTAINER_ID_VALID_CHARACTERS, pos);
+	if (invalid_ch_pos < CONTAINER_ID_LENGTH)


just match the ID against [0-9a-fA-F]{64} ?

This is literally what this code does. Just not using regexes.

speedyguy17 · 2019-02-11T18:37:29Z

userspace/libsinsp/cri.cpp

 #include "sinsp.h"
 #include "sinsp_int.h"

+namespace {
+bool pod_uses_host_netns(const runtime::v1alpha2::PodSandboxStatusResponse& resp)


is this common in the code base instead of static?

I got tired of your (plural you) review comments asking me to switch from static to namespace{}. Make up your mind, will you? :P

yeah i don't really care either way, i was just asking to ensure it's consistent

userspace/libsinsp/cri.cpp

userspace/libsinsp/container_info.h

krishnan-ramkumar · 2019-02-14T00:17:00Z

userspace/libsinsp/container_engine/docker_linux.cpp


+constexpr const cgroup_layout DOCKER_CGROUP_LAYOUT[] = {
+	{"/", ""}, // non-systemd docker
+	{"/docker-", ".scope"}, // systemd docker


How confident are we about the ".scope" suffix of systemd.
this page here:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/resource_management_guide/sec-default_cgroup_hierarchies
seems to suggest we could have ".slice" too.

Will review the entire PR a little later. Wanted to capture this one thing first.

krishnan-ramkumar · 2019-02-14T00:33:15Z

userspace/libsinsp/container_engine/cri.cpp

 bool cri::resolve(sinsp_container_manager* manager, sinsp_threadinfo* tinfo, bool query_os_for_missing_info)
 {
 	sinsp_container_info container_info;

-	if(docker::detect_docker(tinfo, container_info.m_id))
+	if(matches_runc_cgroups(tinfo, CRI_CGROUP_LAYOUT, container_info.m_id))
 	{
 		container_info.m_type = s_cri_runtime_type;


Is the need here only to specify if it is a CRI-based run-time or not ? Do we not need to know if it is CT_CRIO or CT_CONTAINERD ?

yes, we must be as specific as possible. i think we should report a container type being "CRI" only as a last resort.

anoop-sysd · 2019-02-14T02:46:45Z

userspace/libsinsp/container_engine/cri.cpp

@@ -79,32 +109,27 @@ bool parse_cri(sinsp_container_manager *manager, sinsp_container_info *container
 	parse_cri_image(resp_container, container);
 	parse_cri_mounts(resp_container, container);

-	const auto &info_it = resp.info().find("info");
-	if(info_it == resp.info().end())
+	if(parse_containerd(resp, container, tinfo))


this call seems out of place. why call logic specific to containerd unconditionally when the container type is known at this point. if there is an expectation that the logic in parse_containerd() could apply to cri-o as well, why not rename the function to be clear?

btw, can't line 102 be removed since the same operation is done in the caller of this function?

About the duplicate assignment, good catch, removed.

parse_containerd should be pretty cheap when the engine is not containerd compatible (no metadata in info) and if some other runtime decides that this is actually worth following, it saves us two extra api calls.

I'd leave this as is (currently it only applies to containerd and I don't have a better idea for the name).

anoop-sysd · 2019-02-14T02:54:48Z

userspace/libsinsp/container_engine/cri.cpp

 bool cri::resolve(sinsp_container_manager* manager, sinsp_threadinfo* tinfo, bool query_os_for_missing_info)
 {
 	sinsp_container_info container_info;

-	if(docker::detect_docker(tinfo, container_info.m_id))
+	if(matches_runc_cgroups(tinfo, CRI_CGROUP_LAYOUT, container_info.m_id))
 	{
 		container_info.m_type = s_cri_runtime_type;


yes, we must be as specific as possible. i think we should report a container type being "CRI" only as a last resort.

anoop-sysd · 2019-02-14T02:55:34Z

userspace/libsinsp/filterchecks.cpp

@@ -6151,6 +6151,9 @@ uint8_t* sinsp_filter_check_container::extract(sinsp_evt *evt, OUT uint32_t* len
 			case sinsp_container_type::CT_CONTAINERD:
 				m_tstr = "containerd";
 				break;
+			case sinsp_container_type::CT_CRIO:
+				m_tstr = "crio";


Done (in both places). AFAIK this doesn't relate to the backend at all, just to container.type=cri-o style filters and Lua chisels

anoop-sysd · 2019-02-14T02:56:42Z

userspace/libsinsp/chisel_api.cpp

@@ -1193,6 +1193,10 @@ int lua_cbacks::get_container_table(lua_State *ls)
 		{
 			lua_pushstring(ls, "containerd");
 		}
+		else if(it->second.m_type == CT_CRIO)
+		{
+			lua_pushstring(ls, "crio");


anoop-sysd · 2019-02-14T03:21:34Z

@gnosek , i just saw your comment on how the container type is stringified in the backend. so if "cri-o" is not possible, then ignore my comments about renaming "crio".

While containerd returns everything in the `info` dictionary, it's empty for CRIO, which means we need to get the IP address and the image ID from extra API calls. Unfortunately, it seems there's no way to get: * environment variables * privileged status * resource limits in any way from CRIO.

We set the container type in the calling method

gnosek requested review from adalton and mattpag February 5, 2019 19:18

gnosek force-pushed the cri-o branch from d2d8b7f to edca8e8 Compare February 6, 2019 17:03

speedyguy17 suggested changes Feb 11, 2019

View reviewed changes

gnosek force-pushed the cri-o branch from edca8e8 to 2820a68 Compare February 12, 2019 16:11

krishnan-ramkumar reviewed Feb 14, 2019

View reviewed changes

anoop-sysd suggested changes Feb 14, 2019

View reviewed changes

gnosek force-pushed the cri-o branch 2 times, most recently from aa07d75 to e4e4ae4 Compare February 20, 2019 17:45

gnosek force-pushed the cri-o branch from e4e4ae4 to 033b0b8 Compare February 25, 2019 14:32

gnosek added 5 commits February 28, 2019 21:04

CRI-O support

fd056b5

Add CRIO container type

f00262f

Make extra CRI queries configurable (enabled by default)

9004f82

Remove redundant assignment

ad5b647

We set the container type in the calling method

gnosek force-pushed the cri-o branch from 033b0b8 to ad5b647 Compare February 28, 2019 20:30

gnosek merged commit ecafb60 into dev Feb 28, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CRI-O container support #1310

CRI-O container support #1310

gnosek commented Feb 5, 2019

speedyguy17 left a comment

speedyguy17 Feb 11, 2019

gnosek Feb 12, 2019

speedyguy17 Feb 13, 2019

speedyguy17 Feb 11, 2019

gnosek Feb 12, 2019

speedyguy17 Feb 13, 2019

speedyguy17 Feb 11, 2019

speedyguy17 Feb 11, 2019

gnosek Feb 12, 2019

speedyguy17 Feb 13, 2019

speedyguy17 Feb 11, 2019

gnosek Feb 12, 2019

speedyguy17 Feb 13, 2019

speedyguy17 Feb 11, 2019

gnosek Feb 12, 2019

speedyguy17 Feb 13, 2019

speedyguy17 Feb 11, 2019

gnosek Feb 12, 2019

speedyguy17 Feb 11, 2019

gnosek Feb 12, 2019

speedyguy17 Feb 13, 2019

krishnan-ramkumar Feb 14, 2019

krishnan-ramkumar Feb 14, 2019 •

edited

Loading

anoop-sysd Feb 14, 2019

anoop-sysd Feb 14, 2019

anoop-sysd Feb 14, 2019 •

edited

Loading

gnosek Feb 25, 2019

anoop-sysd Feb 14, 2019

anoop-sysd Feb 14, 2019

gnosek Feb 25, 2019

anoop-sysd Feb 14, 2019

anoop-sysd commented Feb 14, 2019


		static_assert(REPORTED_CONTAINER_ID_LENGTH <= CONTAINER_ID_LENGTH, "Reported container ID length cannot be longer than actual length");

		bool detect_one_container_id(const std::string& cgroup, const std::string& prefix, const std::string& suffix, std::string& container_id)

CRI-O container support #1310

CRI-O container support #1310

Conversation

gnosek commented Feb 5, 2019

speedyguy17 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

krishnan-ramkumar Feb 14, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anoop-sysd Feb 14, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anoop-sysd commented Feb 14, 2019

krishnan-ramkumar Feb 14, 2019 •

edited

Loading

anoop-sysd Feb 14, 2019 •

edited

Loading