Add Flink Operator Bakery #21
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yuvipanda
changed the title
Codecov Report
@@ Coverage Diff @@
## main #21 +/- ##
===========================================
- Coverage 98.51% 86.08% -12.44%
===========================================
Files 11 12 +1
Lines 270 309 +39
===========================================
Hits 266 266
- Misses 4 43 +39
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
yuvipanda
force-pushed
the
flink
branch
3 times, most recently
from
745be57 to
6fe22a3
Compare
This needs to: - Create a Flink cluster by setting up a CRD to a kubernetes cluster, as that is where the container image used for execution is specified - Use `kubectl port-forward` to proxy to the flink cluster, so we can actually talk to it This uses https://github.com/apache/flink-kubernetes-operator, as that is the most actively developed, community governed operator. Ref #19
It doesn't just work with *any* Flink cluster, but sets up one Flink cluster per job
Hopefully this can be reached both by whatever is in the k8s cluster as well as code running locally
You can't depend on a non-session fixture in a session fixture. Local IP isn't gonna change per test anyway
Without this, the 'status' field of the CRD is just not populated at all and kubectl wait fully fails
So it doesn't actually try to retrieve 'job_id'. This is gross, and should be fixed
There was an explicit message about kubectl not working due to socat not being found
Could be the reason this doesn't run on GHA
Maybe this is what is needed?
Otherwise, flinkoperator errors
Codecov Report
@@ Coverage Diff @@
## main #21 +/- ##
==========================================
- Coverage 98.51% 98.09% -0.43%
==========================================
Files 11 12 +1
Lines 270 315 +45
==========================================
+ Hits 266 309 +43
- Misses 4 6 +2
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
|
Unfortunately the coverage drop is minor, and for testing the failure modes when kubectl / gcloud isn't present. I'm just going to ignore that and merge, as projeect codecov is only down by |
yuvipanda
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This needs to:
cluster, as that is where the container image used for
execution is specified
kubectl port-forwardto proxy to the flink cluster,so we can actually talk to it
kubectlis installedThis uses https://github.com/apache/flink-kubernetes-operator,
as that is the most actively developed, community governed
operator.
Ref #19