Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Sandbox] k0s #125

Closed
2 tasks done
randybias opened this issue Sep 13, 2024 · 26 comments
Closed
2 tasks done

[Sandbox] k0s #125

randybias opened this issue Sep 13, 2024 · 26 comments
Assignees
Labels

Comments

@randybias
Copy link

randybias commented Sep 13, 2024

Application contact emails

jnummelin@mirantis.com, rbias@mirantis.com

Project Summary

k0s is a CNCF-certified lightweight, Kubernetes distribution with zero dependencies and zero opinion.

Project Description

k0s is a fully conformant, CNCF-certified Kubernetes distribution that offers a zero-dependency, statically compiled, vanilla Kubernetes experience. Designed to run anywhere—from bare metal to cloud environments—k0s provides a minimal footprint with a low attack surface. It enforces a strong separation between controllers and workers, enhancing security and operational efficiency.

k0s aligns with the CNCF mission by fostering an open-source, vendor-neutral Kubernetes distribution that emphasizes choice and flexibility within the cloud-native ecosystem. By providing diversity of choice, k0s encourages innovation and collaboration, strengthening the Kubernetes community.

Org repo URL (provide if all repos under the org are in scope of the application)

https://github.com/k0sproject

Project repo URL in scope of application

https://github.com/k0sproject/k0s

Additional repos in scope of the application

https://github.com/k0sproject/k0sctl

Website URL

https://k0sproject.io/

Roadmap

https://github.com/k0sproject/k0s/milestones

Roadmap context

In our monthly office hours and community channels, we openly discuss roadmap items, gathering input and prioritizing items through collaborative discussions. Our approach is strongly oriented towards fostering a thriving ecosystem, and we actively seek opportunities for interoperability with CNCF projects and other open-source initiatives. This commitment helps us better address the interests and needs of our community.

We are working towards a more formal roadmap.

Contributing Guide

https://github.com/k0sproject/k0s/blob/main/docs/contributors/overview.md

Code of Conduct (CoC)

https://github.com/k0sproject/k0s/blob/main/CODE_OF_CONDUCT.md

Adopters

https://github.com/k0sproject/k0s/blob/main/ADOPTERS.md

Contributing or Sponsoring Org

https://mirantis.com

Maintainers file

https://github.com/k0sproject/k0s/blob/main/MAINTAINERS.md

IP Policy

  • If the project is accepted, I agree the project will follow the CNCF IP Policy

Trademark and accounts

  • If the project is accepted, I agree to donate all project trademarks and accounts to the CNCF

Why CNCF?

k0s seeks to apply to CNCF in order to find a vendor-neutral home. The k0s community has seen significant adoption and has matured rapidly, recently adding additional non-Mirantis maintainers from Replicated.com. With over 7,500 weekly active clusters in the wild that average over 300+ days in age, we believe the time is right to become an official project of the CNCF.

We want to demonstrate our commitment to managing expansion in an open-source manner and ensure neutral governance for our growing community of users and contributors. The CNCF platform will serve as an additional catalyst for community adoption and encourage contributions from developers. Additionally, being a part of CNCF allows k0s to learn from other projects and actively contribute back to the cloud-native community, aligning with the maintainer's ethos.

k0s wants to establish itself as a clear alternative k8s distribution for those who want an ultra lightweight, secure, vanilla distribution that supports all x86 and ARM platforms.

Benefit to the Landscape

The CNCF ecosystem would benefit from having additional conformant k8s distributions, especially ones with a specific niche, such as k0s: lightweight, no opinion, works anywhere and on any platform.

Key Features Include:

Cloud Native 'Fit'

k0s is about as cloud-native as you can get, given it is basically the purest and simplest form of a k8s distribution.

Cloud Native 'Integration'

Nothing besides core Kubernetes.

Cloud Native Overlap

There is some overlap with k3s, but ultimately, we believe there is a enough difference in approach to allow both distributions in the Kubernetes ecosystem.

Here are some different perspectives from around the web for this year, comparing k0s and k3s:

Perhaps more importantly, here are a couple of k0s users explaining why they preferred k0s over k3s:

Similar projects

k3s

Landscape

Yes, k0s is listed in the landscape.

Business Product or Service to Project separation

Some Mirantis products do leverage k0s, but the project is maintained completely separately from our products and has been since inception. Originally Mirantis products used other k8s distributions, but recently migrated to k0s. A close inspection of GitHub issues will show you that there is a clear firewall between k0s and our enterprise products. Putting k0s into the CNCF will further help maintain it's independence.

Project presentations

Runtime TAG in 2021:

https://www.youtube.com/watch?v=5Hja-cGXZ5Q&ab_channel=CNCFTAGRuntime

Edge/IoT WG in 2022:

https://youtu.be/L5jWfM6kn5M?t=1695

Project champions

TBD

Additional information

We collect anonymous usage data. Attached is a screenshot of our k0s dashboard with some detailed statistics showing our accelerated growth over the past year.

k0s Dashboard.pdf

@chris-sanders
Copy link

As a representative of Replicated, I'd like to express our support for the k0s project's application to join the CNCF as a sandbox project.

  1. At Replicated, we have carefully evaluated and selected k0s for our needs, and it has become an integral part of our technology stack.

  2. During our selection process, we reviewed several available projects. What truly sets k0s apart is its commitment to a zero-ops approach. While there are other easy-to-install distributions like k3s, k0s uniquely defines installing, upgrading, and maintaining the cluster as core features within its scope. The built-in Autopilot feature is a prime example of this philosophy, demonstrating a distinctive approach to handling upgrades that's integrated directly into the distribution. This aligns perfectly with our goals of simplifying Kubernetes operations for our users.

  3. We have already implemented k0s in production environments and are seeing great results. Our intention is to continue expanding its use within our infrastructure. We would be delighted to see k0s become part of the CNCF, as we believe this would ensure its continued growth as an open-source community project.

Given our positive experience with k0s and its unique features, we strongly support its inclusion in the CNCF and look forward to its continued development and adoption within the wider community.

@caniszczyk
Copy link
Contributor

Do you have a list of third party dependencies for k0s?

@jnummelin
Copy link

@caniszczyk I'm not sure what you mean here by 3rd party dependencies?

As for any Golang project, our Golang dependencies are managed via go modules: https://github.com/k0sproject/k0s/blob/main/go.mod

The components we embed as binaries are: runc, containerd, Kubernetes (api-server, controller-manages, scheduler, kubelet), kine, etcd, iptables and keepalived
All the embedded bins are statically compiled so we can run them on all Linux OSes and thus have no separate builds or packaging for each distro. The versions for these embedded bins are declared at https://github.com/k0sproject/k0s/blob/main/embedded-bins/Makefile.variables

We also produce SBOM for all the releases: https://github.com/k0sproject/k0s/releases/download/v1.30.4%2Bk0s.0/spdx.json

As for Kubernetes itself, it still has some "3rd party" dependencies, i.e. it does call some tools on the host like mount for example.

@caniszczyk
Copy link
Contributor

@jnummelin this is exactly what I need, essentially we need the runtime dependencies and their associated licenses to ensure they are compliant with CNCF's IP Policy: https://github.com/cncf/foundation/blob/main/allowed-third-party-license-policy.md

@jnummelin
Copy link

jnummelin commented Sep 18, 2024

ok, I've checked the dependency data (attached in CSV format). All the golang modules adhere to the CNCF policy. Some of the embedded binaries, iptables and keepalived, are GPLv2, but IMO they go into this category from the policy:

It is either (A) stored unmodified in a designated third-party folder, or (B) not stored in the CNCF project repository and instead retrieved at installation or build time from the upstream third party repository or package repository

In this case we get the sources for those components directly from upstream repos, build them and "just" include the binary. Here is an example how we do it for iptables, all other components follow the same pattern.

k0s dependencies - Sheet1.csv

@randybias
Copy link
Author

Per @jnummelin:

On Oct 27th k0s crossed another threshold in adoption. We crossed the threshold of seeing 400k unique1 clusters throughout the telemetry collection period starting March 2021. Couldn't be more proud of the k0s team for making k0s what is is today and the dedication to make it better every day.

Onwards and upwards!

Footnotes

  1. We determine unique clusters based on kube-system UUID as k8s uses type 4 UUIDs which makes them globally unique, or at least the chance for collisions is extremely low.

@randybias
Copy link
Author

Replicated recently shared a YouTube video with more details on their use case as well as additional k0s proof points.

https://youtu.be/95LhxbmH-t0?si=4ScS3jDQBRErX1MP

@dims
Copy link
Member

dims commented Dec 20, 2024

@randybias and team, thanks for your submission! I have a couple of questions:

thanks!

@jberkus
Copy link
Contributor

jberkus commented Jan 10, 2025

TAG Contributor strategy has reviewed this project and found the following:

  • The contributor guide is fairly minimal, with some useful detail on Git. (Note that the link in the OP above is incorrect)
  • The project has no written governance, yet.
  • The roadmap is Github milestones, was adopted one release ago and is actively in use
  • There are 8 maintainers, most of whom work for Mirantis and two of whom work for Replicated
  • The project has a moderate amount of public contributor activity, including a few dozen one-time contributors.

This review is for the TOC’s information only. Sandbox projects are not required to have full governance or contributor documentation.

@jnummelin
Copy link

The project has no written governance, yet.

We're setting all this up in a centralized community repo.

Has there been organized efforts to broaden the set of folks who are regular committers?

There is an organized effort to broaden the set of folks who are regular committers by adding personnel to the Open Source Program Office team here at Mirantis which one of the two major maintainer companies to the k0s project. Beyond this the regular community calls have invited contributions externally on issues of focus and there are equal efforts behind k0s (known) users requesting them to contribute on issues they have raised time to time.
k0s usage has mostly been tracked through telemetry data but there has been a dedicated effort starting late last year by community manager @prithvi1307 and team to identify community users and to invite community members to our recently formed #k0s-users and #k0s-dev channels on the k8s slack workspace to contribute back to the project. The project identifies great usage. Around 9k+ active clusters per day and moving ahead is looking to strengthen its adopter count and regular committer count over this year with a focus on future CNCF incubation.

Has there been examples of good back-and-forth between k0s and its upstream (k8s) on issues that matter to your end users?

kubernetes/kubernetes#96115, example of getting rid of dependencies in k8s to better align with k0s goal of "zero dependencies".
kubernetes/kubernetes#116477, fix for armv7 arch which is used by k0s users
kubernetes-sigs/apiserver-network-proxy#363, driven by k0s user request

@randybias
Copy link
Author

Here's an example of an end user in the k0s-dev channel on Kubernetes slack thanking the k0s team for help with getting k0s working with a non-default configuration using NATS and Cilium:

https://kubernetes.slack.com/archives/C07VAPJUECS/p1736700692754159

@jnummelin
Copy link

We just merged the (draft) GTR and security self-assessment docs into the main repo: https://github.com/k0sproject/k0s/tree/main/cncf

@dims
Copy link
Member

dims commented Jan 13, 2025

thanks @randybias and @jnummelin

Also, did you all get a chance to review the telemetry policy for LF projects?
https://www.linuxfoundation.org/legal/telemetry-data-policy

@jnummelin
Copy link

@dims Is that something that needs to be tackled BEFORE the sandbox phase or can it be handled while in sandbox?

@dims
Copy link
Member

dims commented Jan 13, 2025

@jnummelin while in a the sandbox is fine. just surfacing it so you are not surprised.

@srust
Copy link

srust commented Jan 14, 2025

k0s presented to TAG-Runtime on Dec. 19th, 2024: https://www.youtube.com/watch?v=X_i2XIAqKLE

@mrbobbytables
Copy link
Member

/vote

Copy link

git-vote bot commented Jan 14, 2025

Vote created

@mrbobbytables has called for a vote on [Sandbox] k0s (#125).

The members of the following teams have binding votes:

Team
@cncf/cncf-toc

Non-binding votes are also appreciated as a sign of support!

How to vote

You can cast your vote by reacting to this comment. The following reactions are supported:

In favor Against Abstain
👍 👎 👀

Please note that voting for multiple options is not allowed and those votes won't be counted.

The vote will be open for 2months 30days 2h 52m 48s. It will pass if at least 66% of the users with binding votes vote In favor 👍. Once it's closed, results will be published here as a new comment.

@git-vote git-vote bot added the gitvote label Jan 14, 2025
@git-vote git-vote bot added the vote open label Jan 14, 2025
@mrbobbytables
Copy link
Member

@kevin-wangzefeng to followup

@serbrech
Copy link

from this link: https://www.nops.io/blog/k0s-vs-k3s-vs-k8s/

Shared in the description as example of relevant comparison between k0s, k3s and k8s.
I don't know what to make of this... was that just AI generated?

K0s is a container native platform based on distributed systems, such as Apache Kafka and Apache Mesos. It has a strong focus on stream processing and data-driven applications. With a distributed architecture featuring high fault tolerance and scalability, this platform ensures robust performance and availability.

K0s was developed by Google and is used in many of its products, such as Google Ads and Google Photos. This container-native platform was engineered to efficiently run containerized applications in a distributed computing environment. It can manage millions of containers and provide a reliable and scalable platform to handle enterprise-level workloads.

@angellk angellk moved this from 🏗 Upcoming to 🤔 In voting in Sandbox Application Board Jan 15, 2025
@randybias
Copy link
Author

from this link: https://www.nops.io/blog/k0s-vs-k3s-vs-k8s/

Shared in the description as example of relevant comparison between k0s, k3s and k8s. I don't know what to make of this... was that just AI generated?

K0s is a container native platform based on distributed systems, such as Apache Kafka and Apache Mesos. It has a strong focus on stream processing and data-driven applications. With a distributed architecture featuring high fault tolerance and scalability, this platform ensures robust performance and availability.

K0s was developed by Google and is used in many of its products, such as Google Ads and Google Photos. This container-native platform was engineered to efficiently run containerized applications in a distributed computing environment. It can manage millions of containers and provide a reliable and scalable platform to handle enterprise-level workloads.

My bad. Should have read the entire thing. Removed it.

@kevin-wangzefeng
Copy link
Member

The TOC has reviewed the project for sandbox inclusion and suggests the following:

  1. Reach out TAG observability to give an introduction around how you are considering telemetry.
  2. Review and follow the telemetry data collection and usage policy for gathering the data that you mentioned in the application. This can be done during the onboarding period if k0s passes the vote.

@mrbobbytables
Copy link
Member

/check-vote

Copy link

git-vote bot commented Jan 15, 2025

Vote status

So far 72.73% of the users with binding vote are in favor and 0.00% are against (passing threshold: 66%).

Summary

In favor Against Abstain Not voted
8 0 0 3

Binding votes (8)

User Vote Timestamp
dims In favor 2025-01-14 21:29:42.0 +00:00:00
linsun In favor 2025-01-15 3:26:14.0 +00:00:00
TheFoxAtWork In favor 2025-01-14 16:45:12.0 +00:00:00
kgamanji In favor 2025-01-15 8:16:04.0 +00:00:00
angellk In favor 2025-01-14 22:15:40.0 +00:00:00
rochaporto In favor 2025-01-14 21:45:19.0 +00:00:00
nikhita In favor 2025-01-15 2:23:13.0 +00:00:00
kevin-wangzefeng In favor 2025-01-15 13:52:17.0 +00:00:00
@mauilion Pending
@dzolotusky Pending
@cathyhongzhang Pending

Non-binding votes (26)

User Vote Timestamp
adamancini In favor 2025-01-14 16:50:55.0 +00:00:00
laverya In favor 2025-01-14 16:56:24.0 +00:00:00
grantmiller In favor 2025-01-14 16:57:43.0 +00:00:00
Skaronator In favor 2025-01-14 17:09:55.0 +00:00:00
ajp-io In favor 2025-01-14 17:23:07.0 +00:00:00
AmberAlston In favor 2025-01-14 17:32:47.0 +00:00:00
tzabbi In favor 2025-01-14 17:42:27.0 +00:00:00
jtuchscherer In favor 2025-01-14 18:17:57.0 +00:00:00
ramessesii2 In favor 2025-01-14 20:16:17.0 +00:00:00
pschichtel In favor 2025-01-14 22:15:33.0 +00:00:00
iggy In favor 2025-01-14 22:27:48.0 +00:00:00
danielr1996 In favor 2025-01-14 22:38:20.0 +00:00:00
wagnst In favor 2025-01-14 23:22:39.0 +00:00:00
randybias In favor 2025-01-15 0:19:01.0 +00:00:00
CmdrSharp In favor 2025-01-15 1:11:28.0 +00:00:00
serbrech In favor 2025-01-15 2:45:41.0 +00:00:00
bnallapeta In favor 2025-01-15 3:21:25.0 +00:00:00
ferama In favor 2025-01-15 6:19:40.0 +00:00:00
prithvi1307 In favor 2025-01-15 6:38:15.0 +00:00:00
Pogyeah In favor 2025-01-15 7:13:20.0 +00:00:00
Faithcaio In favor 2025-01-15 7:47:11.0 +00:00:00
jnummelin In favor 2025-01-15 9:45:26.0 +00:00:00
adityasonittyl In favor 2025-01-15 10:11:05.0 +00:00:00
pierreozoux In favor 2025-01-15 11:25:48.0 +00:00:00
iapicca In favor 2025-01-15 12:29:13.0 +00:00:00
danmx In favor 2025-01-15 12:56:12.0 +00:00:00

Copy link

git-vote bot commented Jan 16, 2025

Vote closed

The vote passed! 🎉

90.91% of the users with binding vote were in favor and 0.00% were against (passing threshold: 66%).

Summary

In favor Against Abstain Not voted
10 0 0 1

Binding votes (10)

User Vote Timestamp
@dzolotusky In favor 2025-01-15 16:31:18.0 +00:00:00
@kevin-wangzefeng In favor 2025-01-15 13:52:17.0 +00:00:00
@TheFoxAtWork In favor 2025-01-14 16:45:12.0 +00:00:00
@dims In favor 2025-01-14 21:29:42.0 +00:00:00
@linsun In favor 2025-01-15 3:26:14.0 +00:00:00
@nikhita In favor 2025-01-15 2:23:13.0 +00:00:00
@kgamanji In favor 2025-01-15 8:16:04.0 +00:00:00
@rochaporto In favor 2025-01-14 21:45:19.0 +00:00:00
@angellk In favor 2025-01-14 22:15:40.0 +00:00:00
@mauilion In favor 2025-01-15 16:52:34.0 +00:00:00

Non-binding votes (28)

User Vote Timestamp
@adamancini In favor 2025-01-14 16:50:55.0 +00:00:00
@laverya In favor 2025-01-14 16:56:24.0 +00:00:00
@grantmiller In favor 2025-01-14 16:57:43.0 +00:00:00
@Skaronator In favor 2025-01-14 17:09:55.0 +00:00:00
@ajp-io In favor 2025-01-14 17:23:07.0 +00:00:00
@AmberAlston In favor 2025-01-14 17:32:47.0 +00:00:00
@tzabbi In favor 2025-01-14 17:42:27.0 +00:00:00
@jtuchscherer In favor 2025-01-14 18:17:57.0 +00:00:00
@ramessesii2 In favor 2025-01-14 20:16:17.0 +00:00:00
@pschichtel In favor 2025-01-14 22:15:33.0 +00:00:00
@iggy In favor 2025-01-14 22:27:48.0 +00:00:00
@danielr1996 In favor 2025-01-14 22:38:20.0 +00:00:00
@wagnst In favor 2025-01-14 23:22:39.0 +00:00:00
@randybias In favor 2025-01-15 0:19:01.0 +00:00:00
@CmdrSharp In favor 2025-01-15 1:11:28.0 +00:00:00
@serbrech In favor 2025-01-15 2:45:41.0 +00:00:00
@bnallapeta In favor 2025-01-15 3:21:25.0 +00:00:00
@ferama In favor 2025-01-15 6:19:40.0 +00:00:00
@prithvi1307 In favor 2025-01-15 6:38:15.0 +00:00:00
@Pogyeah In favor 2025-01-15 7:13:20.0 +00:00:00
@Faithcaio In favor 2025-01-15 7:47:11.0 +00:00:00
@jnummelin In favor 2025-01-15 9:45:26.0 +00:00:00
@adityasonittyl In favor 2025-01-15 10:11:05.0 +00:00:00
@pierreozoux In favor 2025-01-15 11:25:48.0 +00:00:00
@iapicca In favor 2025-01-15 12:29:13.0 +00:00:00
@danmx In favor 2025-01-15 12:56:12.0 +00:00:00
@emosbaugh In favor 2025-01-15 17:29:48.0 +00:00:00
@p5ntangle In favor 2025-01-15 23:34:19.0 +00:00:00

@mrbobbytables
Copy link
Member

Congrats!
With the vote completed, I've created #333 for following up on sandbox onboarding. I'll go ahead and close this out and further follow up can occur there :)

@github-project-automation github-project-automation bot moved this from 🤔 In voting to ✅ Done in Sandbox Application Board Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

No branches or pull requests