Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Borrow api module from containerd (instead of vendoring it) #3145

Merged
merged 5 commits into from
Aug 1, 2022

Conversation

dims
Copy link
Collaborator

@dims dims commented Jul 29, 2022

We only need the api folder from https://github.com/containerd/containerd project. So we have a script that fetches a zip for a specific version (latest as of now) and sticks it into a third_party/containerd directory. We can run this script as and when needed and check in the code. We do this until containerd/containerd#7231 is fixed on containerd-side.

Note that we patch the imports as well as we are creating a new directory/package structure here as well.

@dims dims force-pushed the borrow-api-module-from-containerd branch 5 times, most recently from 12fe9fd to f82338c Compare July 29, 2022 19:37
@dims
Copy link
Collaborator Author

dims commented Jul 29, 2022

experiment was ✅

Looks like the presubmit(s) are broken because of golang changes. Let's formalize this next week.

@dims
Copy link
Collaborator Author

dims commented Jul 29, 2022

related to containerd/containerd#7231

@akhilerm
Copy link
Contributor

@dims Is this a workaround/hack that we are going to follow for this release?

@dims
Copy link
Collaborator Author

dims commented Jul 30, 2022

@akhilerm yes, that would be my proposal.

@dims dims force-pushed the borrow-api-module-from-containerd branch 3 times, most recently from 42dcb61 to 45d1234 Compare July 30, 2022 17:49
@dims dims changed the title [WIP] Borrow api module from containerd Borrow api module from containerd Jul 30, 2022
@dims dims requested a review from bobbypage July 30, 2022 18:09
@dims dims changed the title Borrow api module from containerd Borrow api module from containerd (instead of vendoring it) Jul 30, 2022
@bobbypage
Copy link
Collaborator

bobbypage commented Aug 1, 2022

Hmm, I'm hitting some issues running the update script myself:

porterdavid@porterdavid: ~/github/cadvisor on borrow-api-module-from-containerd [!?$]
$ gh pr checkout https://github.com/google/cadvisor/pull/3145

porterdavid@porterdavid: ~/github/cadvisor on borrow-api-module-from-containerd [!?$]
$ rm -rf third_party/

porterdavid@porterdavid: ~/github/cadvisor on borrow-api-module-from-containerd [!?$]
$ build/update-containerd-api.sh
~/github/cadvisor/third_party/containerd ~/github/cadvisor
tar: Pattern matching characters used in file names
tar: Use --wildcards to enable pattern matching, or --no-wildcards to suppress this warning
tar: */api: Not found in archive
tar: Pattern matching characters used in file names
tar: Use --wildcards to enable pattern matching, or --no-wildcards to suppress this warning
tar: */LICENSE: Not found in archive
tar: Pattern matching characters used in file names
tar: Use --wildcards to enable pattern matching, or --no-wildcards to suppress this warning
tar: */NOTICE: Not found in archive
tar: Exiting with failure status due to previous errors

@bobbypage
Copy link
Collaborator

Thank you Dims for working on this, I think this will be a great workaround until containerd will have a proper API go module.

go.mod Show resolved Hide resolved
go.mod Show resolved Hide resolved
@dims
Copy link
Collaborator Author

dims commented Aug 1, 2022

@bobbypage what version of tar do you have? if this is on a macos box, it will usually have a really old tar right?

@bobbypage
Copy link
Collaborator

bobbypage commented Aug 1, 2022

@bobbypage what version of tar do you have? if this is on a macos box, it will usually have a really old tar right?

This is on my linux box ~ Debian Testing

$ tar --version
tar (GNU tar) 1.34
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by John Gilmore and Jay Fenlason.

@dims
Copy link
Collaborator Author

dims commented Aug 1, 2022

fixed that wildcards problem @bobbypage (looks like i did this work on my macos which had the older version of tar)

@dims dims force-pushed the borrow-api-module-from-containerd branch 2 times, most recently from 009beda to c508503 Compare August 1, 2022 19:06
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
Signed-off-by: Davanum Srinivas <davanum@gmail.com>
@dims dims force-pushed the borrow-api-module-from-containerd branch from c508503 to 6b2d4b3 Compare August 1, 2022 19:12
@bobbypage
Copy link
Collaborator

LGTM!

@brandond
Copy link

brandond commented Nov 30, 2022

FYI, this has completely broken projects like K3s that try to import both Containerd and Kubelet code into the same project. Containerd and the kubelet's cadvisor end up both registering the same types as part of package init. This was not a problem prior to this commit.

2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.services.containers.v1.Container
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.services.containers.v1.Container.ExtensionsEntry
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.services.containers.v1.Container.LabelsEntry
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.services.containers.v1.Container.Runtime
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.services.containers.v1.GetContainerRequest
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.services.containers.v1.GetContainerResponse
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.services.containers.v1.ListContainersRequest
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.services.containers.v1.ListContainersResponse
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.services.containers.v1.CreateContainerRequest
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.services.containers.v1.CreateContainerResponse
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.services.containers.v1.UpdateContainerRequest
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.services.containers.v1.UpdateContainerResponse
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.services.containers.v1.DeleteContainerRequest
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.services.containers.v1.ListContainerMessage
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.types.Descriptor
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.types.Descriptor.AnnotationsEntry
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.types.Metric
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.types.Mount
2022/11/29 23:49:43 proto: duplicate proto type registered: containerd.types.Platform
panic: proto: duplicate enum registered: containerd.v1.types.Status

goroutine 1 [running]:
github.com/gogo/protobuf/proto.RegisterEnum(...)
	/go/pkg/mod/github.com/gogo/protobuf@v1.3.2/proto/properties.go:520
github.com/google/cadvisor/third_party/containerd/api/types/task.init.0()
	/go/pkg/mod/github.com/google/cadvisor@v0.46.0/third_party/containerd/api/types/task/task.pb.go:161 +0x1a5
	
brandond@dev01:~/go/src/github.com/k3s-io/k3s$ go mod why github.com/google/cadvisor/third_party/containerd/api/types/task
# github.com/google/cadvisor/third_party/containerd/api/types/task
github.com/k3s-io/k3s/pkg/daemons/executor
k8s.io/kubernetes/cmd/kubelet/app
k8s.io/kubernetes/cmd/kubelet/app/options
github.com/google/cadvisor/container/containerd
github.com/google/cadvisor/third_party/containerd/api/types/task

brandond@dev01:~/go/src/github.com/k3s-io/k3s$ go mod why github.com/containerd/containerd/api/types
# github.com/containerd/containerd/api/types
github.com/k3s-io/k3s/pkg/agent/containerd
github.com/containerd/containerd
github.com/containerd/containerd/api/types

@dims
Copy link
Collaborator Author

dims commented Nov 30, 2022

@brandond just to make sure whoever is looking at this. The Kubernetes team does not support the use case of kubelet code imported into other projects/products as mentioned in our README - https://github.com/kubernetes/kubernetes/#to-start-using-k8s pasting the snippet here:

To use Kubernetes code as a library in other applications, see the [list of published components](https://git.k8s.io/kubernetes/staging/README.md). Use of the k8s.io/kubernetes module or k8s.io/kubernetes/... packages as libraries is not supported.

@brandond
Copy link

Yes, we're aware that we are doing naughty things by mashing everything all together and mixing it up with a spoon. It mostly works, except when it doesn't.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants