Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: flux provisioned from init container #208

Merged
merged 68 commits into from
Mar 15, 2024
Merged

Conversation

vsoch
Copy link
Member

@vsoch vsoch commented Oct 10, 2023

Problem: we require flux to be packaged alongside the application logic.
Solution: provide via an init container with an isolated spack build.

@vsoch vsoch changed the title refactor: flux provisioned from sidecar container [WIP] refactor: flux provisioned from sidecar container Oct 10, 2023
Problem: we require flux to be packaged alongside the application logic.
Solution: provide via a sidecar container with an isolated spack build.'

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
also the curve certificate needs to come from a config map so it
is definitely only generated once.

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
@vsoch vsoch changed the title [WIP] refactor: flux provisioned from sidecar container [WIP] refactor: flux provisioned from init container Oct 17, 2023
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Need to debug pip dependencies separately

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
The flux restful api dependencies move too quickly for it to be a reliable
production service, and further, I do not think it should be hard coded
into the operator. It needs to be totally refactored and then
re-added as an example application

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
@vsoch vsoch force-pushed the test-refactor-modular branch 2 times, most recently from 1c01f9d to 0a119ba Compare January 10, 2024 03:18
we can allow the cluster to downsize if the follower broker
exits cleanly with 0, without need for the broker index max
completions attribute that is enabled with a feature gate
and requires k8s 1.28. This change also adds support for a
minSize cluster, which will work to start the quorum when
fewer than the size workers are available. note that this does
not adjust tasks given to a job, so might be assigning too
many tasks to too few workers. This also adds in the previous
downsize workers example, except instead of using pkill for
rockylinux we fall back to flux overlay disconnect, as pkill
is not available by default. It is up to the user to ensure
that the follower broker can be disconnected (and is not running
anything). Finally, we add support for a flux->arch tag,
specifically for an arm binary to be downloaded and used
for the go-wait-fs command.

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
vsoch and others added 11 commits January 26, 2024 23:32
This is tested and working on Google Cloud, and we also
will want to test Aws

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Problem: the Commands.Init was added to replace Commands.Pre, and Commands.Pre
was moved lower. But the original one was not. This means that if the user
defines commands in Commands.Pre, they are run twice.
Solution: remove the top one, as I think I originally intended to do

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Problem: the disable view currently is adding the view bin to the path,
and controller-gen had a release (today, 2/26/2024) where the go.mod is
invalid for go 1.20 (minor version string not supported) and the input
-i is removed. To fix this, we pin the commit from 2 days ago, and
for the view, we ensure we do not change the path if it is being
disabled

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
…roller-gen

bug: openapi-gen and view PATH
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
fix: testing tim updates for generation
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
@vsoch vsoch changed the title [WIP] refactor: flux provisioned from init container refactor: flux provisioned from init container Mar 15, 2024
@vsoch
Copy link
Member Author

vsoch commented Mar 15, 2024

I am supremely tired of having this in this state, and for a review that might be superficial anyway. I'm merging.

@vsoch vsoch merged commit 3caf65c into main Mar 15, 2024
22 checks passed
@vsoch vsoch deleted the test-refactor-modular branch March 15, 2024 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor v0.2.0: Read if you want to use the Flux Operator!
1 participant