K3s Cluster Node(s) Upgrade

[nicklausbrown]

Hello @xunleii,

I did not see a discussions area for the repo, so I will ask my question here as it seems undocumented. Can this terraform module appropriately upgrade a k3s cluster similar to what is described in the "manual" part of the documentation here. If not is there another recommended way to handle version upgrades gracefully?

Otherwise, thanks for open sourcing this project. I haven't finished implementing it yet, but it makes operations a lot easier.

[xunleii]

Hi @nicklausbrown,

I've just checked (and tested) and here the upgrade process using followed by this module :

1. Downloads the latest installation script from Github
2. Reruns the installation script with the new k3s release
3. Waits until the node is ready

I therefore confirm that this module follows the official upgrade procedure by using the official installation script.

However, a word about the upgrade procedure wouldn't go amiss, thanks for bringing it up.

EDIT: replace using by followed by in order to avoid any misunderstanding

[laurentiusoica]

@xunleii During the upgrade process, I noticed that all the nodes get upgraded simultaneously. This leads to a situation where all the master nodes are unavailable for a certain duration. Is there an option to perform the upgrades one after another instead?

[xunleii]

Yep, thank you for that. Indeed, there is no mechanism to have a non-parallel update. I did it because there is little downtime (in most cases) and I didn't want to make this module too complicated.

However, if it's something really blocking, I can make the first server serve as a dependency for the others. With this, there will always be at least one server available to communicate with, but I'll also have to find out if there's a risk of split brain N-1 servers updating at the same time.

[laurentiusoica]

Not exactly a blocker but it breaks the HA guarantees in case of an HA setup. An option to have an one node at a time upgrade would be great.

[laurentiusoica]

@xunleii What are your thoughts on introducing a feature flag for sequential execution that would, when enabled, enforce a sequential creation of servers and agents by adding a depends_on attribute to the preceding resource when the index is greater than 0? When this flag is disabled, which would be the default setting, the current behavior would be maintained.

[xunleii]

@xunleii What are your thoughts on introducing a feature flag for sequential execution that would, when enabled, enforce a sequential creation of servers and agents by adding a depends_on attribute to the preceding resource when the index is greater than 0? When this flag is disabled, which would be the default setting, the current behavior would be maintained.

I agree with you, it can be very useful.
Unfortunately, I don't see how to implement it on Terraform; because it's a module and not a provider, I'm very limited on what I can do. For example here, node resources are created by a loop and all are made in parallel. I don't see how I can make it sequential instead 😞

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If the issue still persists, please leave a comment and it will be reopened.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If the issue still persists, please leave a comment and it will be reopened.