Add less and e3 to the base image

[bart0sh]

Both packages are essential for investigation/debugging and
configuring work. Additional footprint is just 414 kb.

I'm using kind for debugging kubeadm on Mac. It works great!
I've noticed that I always end up installing these packages. 'more' is not good to navigate long outputs.
e3 is the smallest possible editor I could find. Despite being very small it supports WordStar, Emacs, Pico, Nedit and VI command syntax.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: bart0sh
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: munnerz

If they are not already assigned, you can assign the PR to them by writing /assign @munnerz in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[bart0sh]

/cc @BenTheElder

[neolit123]

text reditors are opinionated.
some use nano, same vim, some emacs.

i don't think we should bundle an editor for that reason.

[bart0sh]

/retest

[bart0sh]

how would recommend this for debugging purposes without any editor available in the image?

[bart0sh]

just because they're opinionated it doesn't mean they're not needed.

[neolit123]

just because they're opinionated it doesn't mean they're not needed.

sure but i don't know how to use vi, even.
my goto editor for such editing is nano.

[BenTheElder]

Hmm what about docker exec node cat /bar | less ? (IE use less on the host?)

Regarding vi ... well we need an editor there anyway and this is one of the smallest one

Do we really need an editor on the nodes though? Why not edit on the host? What things are you editing?

Right now the bar for packages on the node is ~ just "things we need to boot a cluster".

I fear the next PR will want emacs /s 😛

[bart0sh]

that's another story. I don't know how to use nano, so we have several choices here:

• do nothing (your suggestion)
• install vi (my suggestion)
• install nano (might be your preference)
• install vi and nano

emacs is out of question due to huge footprint I assume.

Let's ask other people. It's better to satisfy part of the users than none in my opinion.

[BenTheElder]

I hear wanting to debug, but people will probably want different tools for this and I'm not sure we should bundle many if any.

My other concern is if we encourage workflows on the "nodes" people will probably start to depend on that for their CI setups too instead of just having Kubernetes. We already had a few pop up in the wild while kind load ... didn't exist yet.

[bart0sh]

Do we really need an editor on the nodes though? Why not edit on the host? What things are you editing?

configs mainly. I found it quite awkward and inconvenient to switch between host and container just to edit and copy couple of files.

[bart0sh]

My other concern is if we encourage workflows on the "nodes"

I had an impression that Kind is going to be used by developers to debug kubernetes tools. Please, correct me if I'm wrong here. I'm using it exactly for this purpose and I'm very happy about it so far.

[BenTheElder]

I had an impression that Kind is going to be used by developers to debug kubernetes tools. Please, correct me if I'm wrong here. I'm using it exactly for this purpose and I'm very happy about it so far.

That is indeed the primary purpose, but we've picked up a lot of traction for integrating against Kubernetes (eg testing sonobuoy, cluster API) which seems to be an even stronger niche, so considering that is worthwhile. Details are outlined in the docs under project scope.

configs mainly. I found it quite awkward and inconvenient to switch between host and container just to edit and copy couple of files.

Have you seen the ability to add extra mounts? You can bind mount a directory for configs from the host.

[bart0sh]

You can bind mount a directory for configs from the host.

That wouldn't help much if I'd want to edit couple of manifests, systemd drop-ins, some files in /etc etcetera.

[bart0sh]

As I'm compiling on the host anyway (I develop on Mac) I don't need anything else. Just an editor. Of course I can install and do install it. Just thought that I'm not the only one who would benefit from having the opportunity to edit files on the node :)

[BenTheElder]

To be clear this PR overall seems pretty reasonable, I'm just trying to understand your use cases and needs, and I want to avoid opening up to bundling every tool anyone wants for debugging 🙃

Rejecting an emacs PR (if it really is too big) is hard without any line in the sand, and right now the line doesn't encompass this PR.

It also might be worth considering if actually we do need plenty of tools for good debug, in which case we can have both a lite and debug base image.

My debug workflow doesn't need any of these, I poke at the containers from the host generally, but I'm not sure what the typical kubeadm dev needs 😅

[BenTheElder]

With kind build node-image --base-image... we could even make the default base be an extended debug image and then when publishing prebuilt node images we could opt into using the lite image.

Knowing what tools and uses would help figure out if that makes sense. Maybe we should do a survey.

I'd like to bring this up on Monday at our next meeting and see what everyone else thinks. If less and vi suit everyone we can just do this, but if we expect more tools I think something like a debug option for the image might make more sense.

With my own workflow this wasn't even on my radar 😅 I agree that others will probably find something like this useful for debugging kubeadm and probably some of the other components too.

[bart0sh]

To be clear this PR overall seems pretty reasonable, I'm just trying to understand your use cases and needs, and I want to avoid opening up to bundling every tool anyone wants for debugging 🙃

Thank you. I appreciate your attitude.
I'd not call a tiny editor a debugging tool. People would need it for different purposes I believe. Testers, devs, admins - all of them would need an editor to tweak cluster configuration, wouldn't they?

Rejecting an emacs PR (if it really is too big) is hard without any line in the sand, and right now the line doesn't encompass this PR.

Don't worry about that :)

# apt-get install emacs-nox
...
After this operation, 138 MB of additional disk space will be used.

# apt-get install emacs
After this operation, 286 MB of additional disk space will be used.

I'd like to bring this up on Monday at our next meeting and see what everyone else thinks. If less and vi suit everyone we can just do this, but if we expect more tools I think something like a debug option for the image might make more sense.

Sure, let's discuss it. Of course people would want more tools and I don't see anything bad with this. We just need to come up with a criteria for inclusion/rejection.

P.S.: if a footprint is a big concern we can reduce this image. Just a couple of ideas:

root@kind-control-plane:/# du -sh /usr/share/man/
272K	/usr/share/man/
root@kind-control-plane:/# du -sh /usr/share/doc 
3.5M	/usr/share/doc

[aojea]

having ping inside the image will be very useful for troubleshooting network issues too apt-get install iputils-ping

[BenTheElder]

Good ideas for reducing it.

I completely failed to comment back after the meeting, the suggestion there was to create something like:

create images/debug/Dockerfile with:

FROM kindest/base
RUN apt install vi less ping ...

build a base image with:
docker build -t kindest/base:debug images/debug

develop kubernetes with node images built like:
kind build node-image --base-image=kindest/base:debug ...

...

That said, installing a couple of these things in the normal base image is probably reasonable? I'm just not sure where to set the cutoff ... It's very easy to set it at "just what Kubernetes needs" and trim it back at any point based on that.

[tao12345666333]

text reditors are opinionated.
some use nano, same vim, some emacs.

i don't think we should bundle an editor for that reason.

+1 IMO we can provide a manual to tell people how to debug, not to force some way.

I completely failed to comment back after the meeting, the suggestion there was to create something like:

create images/debug/Dockerfile with:

FROM kindest/base
RUN apt install vi less ping ...

build a base image with:
docker build -t kindest/base:debug images/debug

develop kubernetes with node images built like:
kind build node-image --base-image=kindest/base:debug ...

The above can be used as a guide content.

But like the choice of editor or debugging tools, I don't want it to be the focus of our attention, even though I like vi very much.

[bart0sh]

That said, installing a couple of these things in the normal base image is probably reasonable?

I believe it is.

I'm just not sure where to set the cutoff ...

How about setting threshold for a size of the base image and then use 'first come - first served' approach and good old common sense :) ? After all this and similar changes are just PRs and any approver can either approve or reject them.

[bart0sh]

/retest

[alejandrox1]

Throwing in my two cents: I don't like the idea of adding tools to the base image - they are not necessary and if we do then it becomes completely arbitrary what to add and what not to.

I think the idea to add whatever tools a user may ned on top of the node image is the best solution here.

[neolit123]

/hold
no editors please. everybody uses a different one, with different shortcut keys...
a custom Docker image or a script that pulls editors on all nodes is the way to go.

[bart0sh]

@neolit123
it's actually 5 editors in one :)
And editor is even more essential than anything else in my opinion.

Feel free to close this PR. no editor - no interest for me to continue with it.

a custom Docker image or a script that pulls editors on all nodes is the way to go.

That's what I already do, but it's additional work, which could be eliminated, hence this PR.

I agree with @BenTheElder here that it makes sense to have at least some tools in the base image.

[k8s-ci-robot]

@bart0sh: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[BenTheElder]

I've been meaning to setup a possibly compromise for this, just haven't gotten to it yet... we could put a contrib dockerfile like:

ARG BASE_IMAGE=<some default base image version>
FROM ${BASE_IMAGE}
RUN <install editors and other handy debug tools>

With build instructions etc., and then when iterating on kubeadm suggest using that as the --base-image.

[k8s-ci-robot]

@bart0sh: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-kind-conformance-parallel-1-14	c933871	link	/test pull-kind-conformance-parallel-1-14

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[aojea]

I want to share what I'm doing right now for debugging maybe it can be useful, I'm logging into the image and installing my toolset apt-get update && apt-get install vim tcpdump ...
It takes 30 seconds more or less to install them