Consolidate Kubeflow tests #2805

hansinikarunarathne · 2024-07-22T05:45:48Z

…tomize.sh to avoid code repitition

Pull Request Template for Kubeflow manifests Issues

✏️ A brief description of the changes

I created a separate sh file in tests/gh_actions folder to install_KinD , create_KinD_cluster and install_kustomize.sh

🐛 If this PR is related to an issue, please put the link to the issue here.

#2797

✅ Contributor checklist

Make sure you have tested with kustomize. See Installation Prerequisites
All the commits have been signed-off (To pass the DCO check)
Submit the Contributor License Agreements (To pass the cla/google check)

You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

juliusvonkohout · 2024-07-30T10:01:47Z

The goal is consolidate, reduce compexity and the amount of files. We still have as many files as before and it is only adjusted for one higher level workflow.

juliusvonkohout · 2024-07-30T10:42:39Z

tests/gh-actions/install_KinD_create_KinD_cluster_install_kustomize.sh

+
+# Create KinD Cluster
+echo "Creating KinD Cluster..."
+if ! kind create cluster --config tests/gh-actions/kind-cluster.yaml; then


Can we inline all three scripts to save 3 additional files?

I have addressed this. Can you check it? Let me know if you have any concerns.
If not I can delete those redundant files and update the PR. j
Just in the end we have to merge it together.

juliusvonkohout · 2024-07-31T09:44:50Z

.github/workflows/tensorboards_web_application_test.yaml

-      run: ./tests/gh-actions/install_kustomize.sh
+    - name: Install KinD, Create KinD cluster and Install kustomize
+      run: |
+        chmod +x tests/gh-actions/install_KinD_create_KinD_cluster_install_kustomize.sh


no chmod here, you need to fix the file in git itself

I fixed this. If you have any concerns regarding this PR can you let me know? If not I can delete redundant files before the merge

* Expose Ray and Seldon to example kustomization.yaml file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add internal documentations for ray and seldon Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> --------- Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

… 2400s Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

…kubeflow#2819) * Patched PSS labels to multi_tenancy Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added script in gh-actions to patch PSS/static/baseline/pacthes Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added PSS scripts for both baseline and restricted labels of static namespaces and renamed directories Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added tests to enable PSS in gh-actions Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added workflow test for PSS labels Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Fixed indentation Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> --------- Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

biswajit-9776 · 2024-08-17T12:29:35Z

.github/workflows/pipeline_run_from_notebook.yaml

    - tests/gh-actions/install_istio.sh
    - tests/gh-actions/install_cert_manager.sh
    - common/cert-manager/**
-    - common/oauth2-proxy/**
+    - common/oidc-client/oauth2-proxy/**


Do we have common/oidc-client/oauth2-proxy that's replacing common/oauth2-proxy in the paths here?

We have common/oauth2-proxy

biswajit-9776 · 2024-08-18T15:19:22Z

From my end, I ran the latest failing build, and the failing step "Create Kubeflow Notebook with PodDefaults" ran in 300s locally at my end. Only difference I made was creating the cluster using the configuration present in master https://github.com/kubeflow/manifests/blob/master/tests/gh-actions/kind-cluster.yaml instead of this PR. I believe there's something here in Github Actions that's causing this particular fail in this and other PRs.

tzabbi · 2024-08-19T06:29:50Z

Again, can you provide more information? The pipeline creates a notebook resource, but it won't be in the ready state:

kubectl describe Notebook -n kubeflow-user-example-com test

Add this in the file which executes "Create Pipeline Run from Kubeflow Notebook"

juliusvonkohout · 2024-08-19T06:32:37Z

Maybe something regarding caching changed, maybe there are pull rate limits. But I need debugging enabled for more insights.

tzabbi · 2024-08-19T06:37:32Z

Yes you are right that's why I want to describe the resource

biswajit-9776 · 2024-08-19T07:14:52Z

.github/workflows/pipeline_run_from_notebook.yaml

@@ -69,7 +61,7 @@ jobs:
        kubectl apply -f tests/gh-actions/kf-objects/notebook.test.kubeflow-user-example.com.yaml


@hansinikarunarathne after line 61, would be suitable to add the following command before the wait command:

kubectl describe Notebook -n kubeflow-user-example-com test

You can also try this separately in your own PR, this way you can both experiment at the same time.

Yes, I will make a PR to push my change for the cronjob meanwhile.

juliusvonkohout · 2024-08-21T22:17:48Z

@hansinikarunarathne it seems to have been a github issue...
Please change the timeout of Katib and pipeline run from notebook back to 600 seconds and fix the trigger path common/oidc-client/oauth2-proxy/`to common/oauth2-proxy/ in a follow up PR.

/lgtm
/approve

google-oss-prow · 2024-08-21T22:17:55Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: juliusvonkohout

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [juliusvonkohout]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

* Training operator CICD improvements (kubeflow#2779) * Add the networkpolicies Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * rework the training operator tests Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fix the comments Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fix filename Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * try to fix the permissions Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * try to fix the permissions Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * change to the user namespace Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * update the image to rc.1 Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> --------- Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * create seperate file for install_KinD_create_KinD_cluster_install_kustomize.sh to avoid code repitition Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * make sh file executable Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * remove reduntant codes Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * combine kind, kuztomize and kind-cluster create sh files into one file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix issues with combined file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix configurations issues Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix linting issues in workflow files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix trailing issues in workflow files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix trailing issues in workflow file bentoml Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * made tests/gh-actions/install_KinD_create_KinD_cluster_install_kustomize.sh executable and removed chmod command from the workflow files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * made modifications in workflow files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * deleted redundant files from tests folder Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Add kustomize installation file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * made install_kustomize.sh file executable Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * made a fix Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix issue with linting of YAML files (kubeflow#2825) * fix issue with linting of YAML files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * delete a file to check the functionality Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add the after checking the functionality Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add folders to yaml linting Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * check linting issues Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * check linting issues Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * check linting changes Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * check linting functionality by chnaging different files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made some fixes in linting YAML file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made some fixes in linting YAML file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made some fixes in linting YAML file and check the functionality Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made some fixes in linting YAML file and check the functionality Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add github, hack and tests folders also for linting Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Check functionality by deleting sh from hack and yaml file common Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Added deleted files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> --------- Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Expose Ray and Seldon to example kustomization.yaml file (kubeflow#2834) * Expose Ray and Seldon to example kustomization.yaml file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add internal documentations for ray and seldon Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> --------- Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Increase the time out of notebook and katib test Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * increase the timout time Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * increase timeout of the create pipeline Run from Kubeflow Notebook to 2400s Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Add code to calculate time taken to pull a docker image Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix syntax error in yaml file Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * increase pipeline time Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * remove one step in action Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Added tests to tests/gh-actions to enable baseline and restricted PSS (kubeflow#2819) * Patched PSS labels to multi_tenancy Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added script in gh-actions to patch PSS/static/baseline/pacthes Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added PSS scripts for both baseline and restricted labels of static namespaces and renamed directories Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added tests to enable PSS in gh-actions Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added workflow test for PSS labels Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Fixed indentation Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> --------- Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * add kind cluster step into pss test yaml file Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * change time pot time Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> --------- Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> Co-authored-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> Co-authored-by: biswajit-9776 <115724497+biswajit-9776@users.noreply.github.com>

* Training operator CICD improvements (kubeflow#2779) * Add the networkpolicies Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * rework the training operator tests Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fix the comments Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fix filename Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * try to fix the permissions Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * try to fix the permissions Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * change to the user namespace Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * update the image to rc.1 Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> --------- Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * create seperate file for install_KinD_create_KinD_cluster_install_kustomize.sh to avoid code repitition Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * make sh file executable Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * remove reduntant codes Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * combine kind, kuztomize and kind-cluster create sh files into one file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix issues with combined file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix configurations issues Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix linting issues in workflow files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix trailing issues in workflow files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix trailing issues in workflow file bentoml Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * made tests/gh-actions/install_KinD_create_KinD_cluster_install_kustomize.sh executable and removed chmod command from the workflow files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * made modifications in workflow files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * deleted redundant files from tests folder Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Add kustomize installation file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * made install_kustomize.sh file executable Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * made a fix Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix issue with linting of YAML files (kubeflow#2825) * fix issue with linting of YAML files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * delete a file to check the functionality Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add the after checking the functionality Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add folders to yaml linting Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * check linting issues Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * check linting issues Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * check linting changes Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * check linting functionality by chnaging different files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made some fixes in linting YAML file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made some fixes in linting YAML file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made some fixes in linting YAML file and check the functionality Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made some fixes in linting YAML file and check the functionality Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add github, hack and tests folders also for linting Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Check functionality by deleting sh from hack and yaml file common Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Added deleted files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> --------- Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Expose Ray and Seldon to example kustomization.yaml file (kubeflow#2834) * Expose Ray and Seldon to example kustomization.yaml file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add internal documentations for ray and seldon Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> --------- Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Increase the time out of notebook and katib test Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * increase the timout time Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * increase timeout of the create pipeline Run from Kubeflow Notebook to 2400s Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Add code to calculate time taken to pull a docker image Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix syntax error in yaml file Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * increase pipeline time Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * remove one step in action Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Added tests to tests/gh-actions to enable baseline and restricted PSS (kubeflow#2819) * Patched PSS labels to multi_tenancy Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added script in gh-actions to patch PSS/static/baseline/pacthes Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added PSS scripts for both baseline and restricted labels of static namespaces and renamed directories Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added tests to enable PSS in gh-actions Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added workflow test for PSS labels Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Fixed indentation Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> --------- Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * add kind cluster step into pss test yaml file Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * change time pot time Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> --------- Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> Co-authored-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> Co-authored-by: biswajit-9776 <115724497+biswajit-9776@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

* Training operator CICD improvements (kubeflow#2779) * Add the networkpolicies Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * rework the training operator tests Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fix the comments Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fix filename Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * try to fix the permissions Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * try to fix the permissions Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * change to the user namespace Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * update the image to rc.1 Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> * fixes Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> --------- Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * create seperate file for install_KinD_create_KinD_cluster_install_kustomize.sh to avoid code repitition Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * make sh file executable Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * remove reduntant codes Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * combine kind, kuztomize and kind-cluster create sh files into one file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix issues with combined file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix configurations issues Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix linting issues in workflow files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix trailing issues in workflow files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix trailing issues in workflow file bentoml Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * made tests/gh-actions/install_KinD_create_KinD_cluster_install_kustomize.sh executable and removed chmod command from the workflow files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * made modifications in workflow files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * deleted redundant files from tests folder Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Add kustomize installation file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * made install_kustomize.sh file executable Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * made a fix Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix issue with linting of YAML files (kubeflow#2825) * fix issue with linting of YAML files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * delete a file to check the functionality Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add the after checking the functionality Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add folders to yaml linting Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * check linting issues Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * check linting issues Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * check linting changes Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * check linting functionality by chnaging different files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made some fixes in linting YAML file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made some fixes in linting YAML file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made some fixes in linting YAML file and check the functionality Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * made some fixes in linting YAML file and check the functionality Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add github, hack and tests folders also for linting Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Check functionality by deleting sh from hack and yaml file common Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Added deleted files Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> --------- Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Expose Ray and Seldon to example kustomization.yaml file (kubeflow#2834) * Expose Ray and Seldon to example kustomization.yaml file Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> * Add internal documentations for ray and seldon Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> --------- Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Increase the time out of notebook and katib test Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * increase the timout time Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * increase timeout of the create pipeline Run from Kubeflow Notebook to 2400s Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Add code to calculate time taken to pull a docker image Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * fix syntax error in yaml file Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * increase pipeline time Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * remove one step in action Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * Added tests to tests/gh-actions to enable baseline and restricted PSS (kubeflow#2819) * Patched PSS labels to multi_tenancy Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added script in gh-actions to patch PSS/static/baseline/pacthes Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added PSS scripts for both baseline and restricted labels of static namespaces and renamed directories Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added tests to enable PSS in gh-actions Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Added workflow test for PSS labels Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> * Fixed indentation Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> --------- Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * add kind cluster step into pss test yaml file Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> * change time pot time Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> --------- Signed-off-by: juliusvonkohout <45896133+juliusvonkohout@users.noreply.github.com> Signed-off-by: hansinikarunarathne <107214435+hansinikarunarathne@users.noreply.github.com> Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk> Signed-off-by: biswajit-9776 <biswajitpatt139@gmail.com> Co-authored-by: Julius von Kohout <45896133+juliusvonkohout@users.noreply.github.com> Co-authored-by: biswajit-9776 <115724497+biswajit-9776@users.noreply.github.com> Signed-off-by: Patrick Schönthaler <patrick.schoenthaler@itsc.de>

google-oss-prow bot requested review from juliusvonkohout and kimwnasptd July 22, 2024 05:45

google-oss-prow bot added the size/M label Jul 22, 2024

google-oss-prow bot added size/L size/XXL and removed size/M size/L labels Jul 30, 2024

hansinikarunarathne force-pushed the organize_kubeflow_tests branch from 75789b6 to ee570fb Compare July 30, 2024 09:47

google-oss-prow bot added size/L and removed size/XXL labels Jul 30, 2024

hansinikarunarathne force-pushed the organize_kubeflow_tests branch from ee570fb to b84f222 Compare July 30, 2024 09:51

google-oss-prow bot added size/M and removed size/L labels Jul 30, 2024

juliusvonkohout self-assigned this Jul 30, 2024

google-oss-prow bot added size/L and removed size/M labels Jul 30, 2024

juliusvonkohout changed the title ~~Organize Kubeflow tests~~ Consolidate Kubeflow tests Jul 30, 2024

juliusvonkohout reviewed Jul 30, 2024

View reviewed changes

juliusvonkohout linked an issue Jul 30, 2024 that may be closed by this pull request

Reorganize Kubeflow tests #2797

Open

7 tasks

google-oss-prow bot added size/XL and removed size/L labels Jul 30, 2024

hansinikarunarathne force-pushed the organize_kubeflow_tests branch from e8938f7 to c7e3bf9 Compare July 30, 2024 15:53

google-oss-prow bot added size/L size/XL size/XXL and removed size/XL size/L labels Jul 30, 2024

juliusvonkohout reviewed Jul 31, 2024

View reviewed changes

hansinikarunarathne and others added 9 commits August 15, 2024 00:28

Increase the time out of notebook and katib test

159abc9

Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

increase the timout time

08e8f57

Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

increase timeout of the create pipeline Run from Kubeflow Notebook to…

e847d4f

… 2400s Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

Add code to calculate time taken to pull a docker image

6de8ebc

Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

fix syntax error in yaml file

3227d17

Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

increase pipeline time

33c0419

Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

remove one step in action

12b1375

Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

hansinikarunarathne force-pushed the organize_kubeflow_tests branch from 58ad7ee to d925710 Compare August 14, 2024 18:59

hansinikarunarathne and others added 2 commits August 15, 2024 00:32

add kind cluster step into pss test yaml file

3702500

Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

Merge branch 'master' into organize_kubeflow_tests

c2d20ef

google-oss-prow bot added size/L and removed size/XL labels Aug 14, 2024

change time pot time

087c017

Signed-off-by: hansinikarunarathne <hansini.20@cse.mrt.ac.lk>

biswajit-9776 reviewed Aug 17, 2024

View reviewed changes

biswajit-9776 reviewed Aug 19, 2024

View reviewed changes

google-oss-prow bot added the lgtm label Aug 21, 2024

google-oss-prow bot added the approved label Aug 21, 2024

google-oss-prow bot merged commit e04bd39 into kubeflow:master Aug 21, 2024
29 checks passed

juliusvonkohout mentioned this pull request Aug 21, 2024

Reorganize Kubeflow tests #2797

Open

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Consolidate Kubeflow tests #2805

Consolidate Kubeflow tests #2805

hansinikarunarathne commented Jul 22, 2024

juliusvonkohout commented Jul 30, 2024 •

edited

Loading

juliusvonkohout Jul 30, 2024

hansinikarunarathne Jul 30, 2024 •

edited by juliusvonkohout

Loading

juliusvonkohout Jul 31, 2024

hansinikarunarathne Aug 2, 2024

biswajit-9776 Aug 17, 2024

juliusvonkohout Aug 18, 2024

biswajit-9776 commented Aug 18, 2024

tzabbi commented Aug 19, 2024

juliusvonkohout commented Aug 19, 2024 •

edited

Loading

tzabbi commented Aug 19, 2024

biswajit-9776 Aug 19, 2024

juliusvonkohout Aug 19, 2024

biswajit-9776 Aug 19, 2024 •

edited

Loading

juliusvonkohout commented Aug 21, 2024

google-oss-prow bot commented Aug 21, 2024

		@@ -69,7 +61,7 @@ jobs:
		kubectl apply -f tests/gh-actions/kf-objects/notebook.test.kubeflow-user-example.com.yaml

Consolidate Kubeflow tests #2805

Consolidate Kubeflow tests #2805

Conversation

hansinikarunarathne commented Jul 22, 2024

Pull Request Template for Kubeflow manifests Issues

✏️ A brief description of the changes

🐛 If this PR is related to an issue, please put the link to the issue here.

✅ Contributor checklist

juliusvonkohout commented Jul 30, 2024 • edited Loading

juliusvonkohout Jul 30, 2024

Choose a reason for hiding this comment

hansinikarunarathne Jul 30, 2024 • edited by juliusvonkohout Loading

Choose a reason for hiding this comment

juliusvonkohout Jul 31, 2024

Choose a reason for hiding this comment

hansinikarunarathne Aug 2, 2024

Choose a reason for hiding this comment

biswajit-9776 Aug 17, 2024

Choose a reason for hiding this comment

juliusvonkohout Aug 18, 2024

Choose a reason for hiding this comment

biswajit-9776 commented Aug 18, 2024

tzabbi commented Aug 19, 2024

juliusvonkohout commented Aug 19, 2024 • edited Loading

tzabbi commented Aug 19, 2024

biswajit-9776 Aug 19, 2024

Choose a reason for hiding this comment

juliusvonkohout Aug 19, 2024

Choose a reason for hiding this comment

biswajit-9776 Aug 19, 2024 • edited Loading

Choose a reason for hiding this comment

juliusvonkohout commented Aug 21, 2024

google-oss-prow bot commented Aug 21, 2024

juliusvonkohout commented Jul 30, 2024 •

edited

Loading

hansinikarunarathne Jul 30, 2024 •

edited by juliusvonkohout

Loading

juliusvonkohout commented Aug 19, 2024 •

edited

Loading

biswajit-9776 Aug 19, 2024 •

edited

Loading