Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubeflow 1.4->1.6 Rebase #99

Merged
merged 209 commits into from
Sep 20, 2022
Merged

Kubeflow 1.4->1.6 Rebase #99

merged 209 commits into from
Sep 20, 2022

Conversation

bryanpaget
Copy link

@bryanpaget bryanpaget commented Sep 15, 2022

Closes StatCan/aaw#1341

Thank you @brendangadd, @Souheil-Yazji for the following instructions on how to rebase:

git remote add upstream https://github.com/kubeflow/kubeflow/
git checkout upstream/master # make sure you have the forked repo's master locally
git checkout stc-master
git pull
git checkout -b {your-new-branch-to-rebase-fork}
git reset $(git merge-base upstream/master $(git branch --show-current)) # this will reset your current branch to a good merge base between it and the upstream master, essentially the idea is to undo all commits not on upstream master
git add .
git commit -m "commit message"
git rebase upstream/v1.6.0

I tried to stay upstream as much as possible.

The folders I tended to take our changes from were:

components/centraldashboard
components/notebook-controller
.github/workflows

Testing

npm run test client

Testing the client gives:

TOTAL: 1 FAILED, 65 SUCCESS

npm run test-server

Testing the server gives:

Summary:

👊  Passed
Suites:  19 of 19
Specs:   45 of 45
Expects: 111 (0 failures)
Finished in 0.393 seconds

See below for test output.

yanniszark and others added 30 commits March 25, 2021 18:37
Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>
Removing myself as an owner. This should lead to better auto-assignment of code reviews.
* jwa(front): Add npm rule for checking the format

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* ci: Add common tasks for format checks

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* ci(jwa): Add format check tasks

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* jwa(back): Fix formatting

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* jwa(front): Fix formatting

Run `npm run format:write` on frontend

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* jwa(front): Include prettier in package.json

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* jwa(docker): Don't copy node_modules in Dockerfile

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* web-apps: Add global dockerignore file

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* jwa: Remove unused dockerignore file

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* jwa(make): Don't include dockerignore and cleanup

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
When the TB controller attempts to schedule a RWO PVC it checks its
accessModes in the PVC status. The controller panics if the list is
empty.

This commit adds a check to ensure the list is not empty.

Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
* api: Add marker for preserving unknown fields in Plugins

Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>

* manifests: Regenerate manifests

Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>
…urable (kubeflow#5823)

* rebase: Make logos configurable in configmap and remove trademark references

Rebased to remove the changes to the package-lock.json

* review: add suggested changes and add image group section to README
…low#5815)

* Make notebook limits configurable with a multiplication factor

* Make limits configurable under advanced section

* run prettier to format frontend code

* fix formatting and add rounding in backend

* Return error if limit is smaller than request

* Allow disabling limitFactor by setting it to none

* review: remove camelCase in python backend

* fix: update spawner_ui_config.yaml in manifests directory

* review: fix setting limits backend

* review: remove unnecessary check from backend
* vwa(front): Add npm script to check the formatting

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* vwa(front): Update the package-lock.json

Run `npm install` to bring the package-lock.json up to date

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* vwa(front): Fix formatting

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* vwa(back): Fix formatting

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* vwa(docker): Remove unused dockerignore file

We have created a global dockerignore file for all the web apps in the
parent dir.

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* vwa(docker): Don't copy node_modules in dockerfile

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* vwa(make): Don't include dockerignore

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* ci(vwa): Add format check tasks

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* jwa(docker): Copy only necessary files for build

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* review: Use docker context instead of cd ..

Don't use a `cd ..` and copy dockerignore files back and forth. Instead
we should use the Docker context and the global dockerignore file we
have for all the web apps.

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* jwa(front): Add static logos in the app

The app does not contain the logos' svgs in its source code/static
files. This results in the icons to not show when developing locally.

This commit adds the svgs found in the logos ConfigMap to the static
files of the app as well.

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* jwa(front): Change logos fetch url

Change the URLs of the logos from `static/assets/*` to
`static/assets/logos`.

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>

* jwa(manifests): Don't override assets with logos

Mount the ConfigMap under the `static/assets/logos` directory to not
override the contents of the entire assets dir.

Signed-off-by: Kimonas Sotirchos <kimwnasptd@arrikto.com>
* Make it possible to add Namespaced menu items

* Reduce the nesting levels
…ubeflow#5891)

* Update form.py

* Update form.py

* Update form-default.component.ts

* fix whitespace issue
* cwa(front): Ignore font files in assets

* feat(jupyter): add fonts as assets to service

* CRUD: fonts in common

* CWA: Remove link to css file

* jwa(front): Remove font assets from jupyter

Co-authored-by: Wendy Gaultier <wvgaultier@gmail.com>
* Update base notebook for Renovate compatibility

* Separate kubectl and S6 arch
…n iframe (kubeflow#5961)

* Update jupyter, rstudio and vs code notebook images

* Remove sed and use substring removal
…w#5952)

* [fix]: Make jupyter-web-app parse workspace volume MountPath

- workspace volume path was fixed with "/home/jovyan"
- it should be enable to parse from jupyter-web-app-config's data

* change parsing key correctly
Knative control-plane components need to probe certain paths of Knative
Pods, in order to function correctly. These paths are:
- /healthz
- /metrics
- /ready
- /wait-for-drain

For this reason, we extend the Profile Controller to apply an Istio
AuthorizationPolicy that allows traffic to these HTTP paths for Pods in
user namespaces, as per Knative's instructions:
https://knative.dev/docs/serving/istio-authorization/#allowing-access-from-system-pods-by-paths

This only fixes requests through the public ingress. Cluster-internal
traffic and predictor-transformer use-cases are not supported yet. We
continue discussing these in kubeflow#5965.

Refs kubeflow#5965
Refs kserve/kserve#1558

Signed-off-by: Yurii Komar <subreptivus@gmail.com>
* feat(jupyter): Support hiding image registry/tag

* feat(jupyter): Support hiding VSCode/RStudio images registry/tag

* rebasing onto upstream master

* fix: formatting with prettier and set default hideRegistry=true

* fix: move hideRegistry and hideTag from spawnerFormDefaults.image

* fix whitespace

* add tooltip for images

Co-authored-by: JessicaBarh <jessicabarhoma@gmail.com>
* Update dockerfiles and make compatible with Renovate

* Set memory for jupyter pytorch to the same as jupyter tensorflow

* Update protobuf

* Remove conda version and use substring expansion

* Update SQLAlchemy

* Update dill
Bryan Paget added 2 commits September 13, 2022 13:30
# Conflicts:
#	.github/workflows/build-centraldashboard.yml
#	.github/workflows/build-notebookcontroller.yml
#	OWNERS
#	ROADMAP.md
#	components/admission-webhook/go.mod
#	components/admission-webhook/go.sum
#	components/admission-webhook/manifests/base/crd.yaml
#	components/admission-webhook/manifests/base/kustomization.yaml
#	components/centraldashboard/Dockerfile
#	components/centraldashboard/package-lock.json
#	components/centraldashboard/package.json
#	components/centraldashboard/public/components/main-page.js
#	components/crud-web-apps/common/frontend/kubeflow-common-lib/package-lock.json
#	components/crud-web-apps/common/frontend/kubeflow-common-lib/package.json
#	components/crud-web-apps/common/frontend/kubeflow-common-lib/projects/kubeflow/src/lib/form/positive-number-input/positive-number-input.component.html
#	components/crud-web-apps/common/frontend/kubeflow-common-lib/projects/kubeflow/src/lib/resource-table/resource-table.module.ts
#	components/crud-web-apps/common/frontend/kubeflow-common-lib/projects/kubeflow/src/lib/resource-table/table/table.component.ts
#	components/crud-web-apps/common/frontend/kubeflow-common-lib/projects/kubeflow/src/public-api.ts
#	components/crud-web-apps/jupyter/Dockerfile
#	components/crud-web-apps/jupyter/Makefile
#	components/crud-web-apps/jupyter/README.md
#	components/crud-web-apps/jupyter/backend/apps/common/form.py
#	components/crud-web-apps/jupyter/backend/apps/common/yaml/spawner_ui_config.yaml
#	components/crud-web-apps/jupyter/backend/apps/default/routes/post.py
#	components/crud-web-apps/jupyter/frontend/.gitignore
#	components/crud-web-apps/jupyter/frontend/angular.json
#	components/crud-web-apps/jupyter/frontend/e2e/tsconfig.json
#	components/crud-web-apps/jupyter/frontend/i18n/fr/messages.fr.xlf
#	components/crud-web-apps/jupyter/frontend/i18n/messages.xlf
#	components/crud-web-apps/jupyter/frontend/package-lock.json
#	components/crud-web-apps/jupyter/frontend/package.json
#	components/crud-web-apps/jupyter/frontend/src/app/app.component.spec.ts
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/form-advanced-options/form-advanced-options.component.html
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/form-affinity-tolerations/form-affinity-tolerations.component.html
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/form-configurations/form-configurations.component.html
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/form-cpu-ram/form-cpu-ram.component.html
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/form-cpu-ram/form-cpu-ram.component.ts
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/form-data-volumes/form-data-volumes.component.html
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/form-default.component.html
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/form-default.component.ts
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/form-gpus/form-gpus.component.html
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/form-gpus/form-gpus.component.ts
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/form-image/form-image.component.html
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/form-name/form-name.component.html
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/form-workspace-volume/form-workspace-volume.component.html
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/form-workspace-volume/form-workspace-volume.component.ts
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/utils.ts
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/volume/volume.component.html
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-default/volume/volume.component.spec.ts
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-rok/form-rok.component.html
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-rok/rok-form-data-volumes/rok-form-data-volumes.component.spec.ts
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-rok/rok-form-data-volumes/rok-form-data-volumes.component.ts
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-rok/rok-form-workspace-volume/rok-form-workspace-volume.component.spec.ts
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-rok/rok-volume/rok-volume.component.spec.ts
#	components/crud-web-apps/jupyter/frontend/src/app/pages/form/form-rok/utils.ts
#	components/crud-web-apps/jupyter/frontend/src/app/pages/index/index-default/config.ts
#	components/crud-web-apps/jupyter/frontend/src/app/types.ts
#	components/crud-web-apps/jupyter/frontend/src/proxy.conf.json
#	components/crud-web-apps/jupyter/manifests/base/configs/spawner_ui_config.yaml
#	components/crud-web-apps/jupyter/manifests/base/kustomization.yaml
#	components/crud-web-apps/tensorboards/frontend/package-lock.json
#	components/crud-web-apps/tensorboards/frontend/package.json
#	components/crud-web-apps/tensorboards/frontend/src/app/pages/index/config.ts
#	components/crud-web-apps/tensorboards/manifests/base/kustomization.yaml
#	components/crud-web-apps/volumes/Dockerfile
#	components/crud-web-apps/volumes/Makefile
#	components/crud-web-apps/volumes/frontend/package-lock.json
#	components/crud-web-apps/volumes/frontend/package.json
#	components/crud-web-apps/volumes/frontend/src/app/pages/index/config.ts
#	components/crud-web-apps/volumes/manifests/base/kustomization.yaml
#	components/example-notebook-servers/README.md
#	components/example-notebook-servers/base/Dockerfile
#	components/example-notebook-servers/codeserver-python/Dockerfile
#	components/example-notebook-servers/codeserver/Dockerfile
#	components/example-notebook-servers/jupyter-pytorch-full/cpu.Dockerfile
#	components/example-notebook-servers/jupyter-pytorch-full/cuda.Dockerfile
#	components/example-notebook-servers/jupyter-pytorch/cpu.Dockerfile
#	components/example-notebook-servers/jupyter-pytorch/cuda.Dockerfile
#	components/example-notebook-servers/jupyter-scipy/Dockerfile
#	components/example-notebook-servers/jupyter-tensorflow-full/cpu.Dockerfile
#	components/example-notebook-servers/jupyter-tensorflow-full/cuda.Dockerfile
#	components/example-notebook-servers/jupyter-tensorflow/cpu-requirements.txt
#	components/example-notebook-servers/jupyter-tensorflow/cpu.Dockerfile
#	components/example-notebook-servers/jupyter-tensorflow/cuda.Dockerfile
#	components/example-notebook-servers/jupyter/Dockerfile
#	components/example-notebook-servers/jupyter/requirements.txt
#	components/example-notebook-servers/rstudio-tidyverse/Dockerfile
#	components/example-notebook-servers/rstudio/Dockerfile
#	components/notebook-controller/config/base/kustomization.yaml
#	components/notebook-controller/controllers/notebook_controller.go
#	components/profile-controller/config/base/kustomization.yaml
#	components/profile-controller/config/base/namespace-labels.yaml
#	components/profile-controller/config/overlays/kubeflow/kustomization.yaml
#	components/profile-controller/controllers/profile_controller.go
#	components/profile-controller/controllers/profile_controller_test.go
#	components/profile-controller/go.mod
#	components/profile-controller/go.sum
#	components/profile-controller/main.go
#	components/tensorboard-controller/config/base/kustomization.yaml
#	components/tensorboard-controller/config/crd/bases/tensorboard.kubeflow.org_tensorboards.yaml
#	components/tensorboard-controller/go.mod
#	components/tensorboard-controller/go.sum
#	releasing/update-manifests-images
#	releasing/version/VERSION
@bryanpaget
Copy link
Author

Linked Issue: StatCan/aaw#1341

@Jose-Matsuda
Copy link

Jose-Matsuda commented Sep 16, 2022

Seems we lost a commit from Saffa somewhere along the way
image

See 8de6feb , I cannot seem to find these lines of code anymore.

Having said this, this line of code may need to change anyways with how we are changing how the logic around SAS works, may want to confirm with the team about this, may need to look at the other related files in her PR as well to make sure that her changes are still in here.

Copy link

@Jose-Matsuda Jose-Matsuda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Address the comments made and I think we good

@bryanpaget
Copy link
Author

Seems we lost a commit from Saffa somewhere along the way image

See 8de6feb , I cannot seem to find these lines of code anymore.

Having said this, this line of code may need to change anyways with how we are changing how the logic around SAS works, may want to confirm with the team about this, may need to look at the other related files in her PR as well to make sure that her changes are still in here.

I've added it back! :-)

@Jose-Matsuda
Copy link

Seems we lost a commit from Saffa somewhere along the way image
See 8de6feb , I cannot seem to find these lines of code anymore.
Having said this, this line of code may need to change anyways with how we are changing how the logic around SAS works, may want to confirm with the team about this, may need to look at the other related files in her PR as well to make sure that her changes are still in here.

I've added it back! :-)

Ooops we still missing a tiny bit inside the languages.json file.

image

Some fixes were lost in the rebase. They have been added back.
Copy link

@Jose-Matsuda Jose-Matsuda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks better now i think

@chuckbelisle chuckbelisle merged commit b9f455d into stc-master Sep 20, 2022
@Souheil-Yazji Souheil-Yazji changed the title Feature 1.6 upgrade Kubeflow 1.4->1.6 Rebase Sep 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rebase Kubeflow onto v1.6.0