Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Storage management documentation update #266

Merged
merged 73 commits into from
Nov 28, 2024
Merged

Storage management documentation update #266

merged 73 commits into from
Nov 28, 2024

Conversation

BluValor
Copy link
Collaborator

@BluValor BluValor commented Nov 21, 2024 •
edited
Loading

Fixes / Features

  • add example for gcs fuse manifest
  • add additional info to troubleshooting

This PR is dependent on PR #218, merge after #218.

Testing / Documentation

Testing details.

  • [ y/n ] Tests pass
  • [ y/n ] Appropriate changes to documentation are included in the PR

pawloch00 and others added 30 commits October 23, 2024 09:03
@pawloch00
add flag parser and enable GCPFilestore on cluster
0ecf53c 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
update cluster with gcp filestore
8491a43 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
fix linter
f121fa9 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
filestore -> gcpfilestore
0bc8636 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
filetore -> gcpfilestorage
9679b7b 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
create filestore instance
fef30a3 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
add enable filestore
1b45628 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
enable gcpfilestore on cluster creation
80ae221 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
pyink
89c3cd0 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
add to workload
f67eda7 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
pyink
f66627f 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
Merge branch 'ppawl-create-gcpfilestore-instance' of https://github.c…
6077eca 
…om/AI-Hypercomputer/xpk into ppawl-use-filestore-workload
@pawloch00
fix linter
f94fe6f 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
Merge branch 'ppawl-filestore-service-account' into ppawl-create-gcpf…
4a6bd19 
…ilestore-instance
@pawloch00
fix string formatting
729546c 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
Merge branch 'ppawl-filestore-service-account' of https://github.com/…
f5a7479 
…AI-Hypercomputer/xpk into ppawl-create-gcpfilestore-instance
@pawloch00
apply manifest file
93c72c1 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
Merge branch 'ppawl-create-gcpfilestore-instance' of https://github.c…
f82d795 
…om/AI-Hypercomputer/xpk into ppawl-use-filestore-workload
@pawloch00
add workloads
e39e894 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@PBundyra
Fix get storage function
fa7a49d
@pawloch00
Merge branch 'development' of https://github.com/AI-Hypercomputer/xpk …
f405ba6 
…into ppawl-use-filestore-workload
@pawloch00
pyink;
b72b707 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
remove adding iam
072530a 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
rm kjob
2baf795 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
rm examples
32975e4 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
rm duplicated
9b4586f 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
update filestore iam
3da11da 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
Merge branch 'storage-fix' of https://github.com/AI-Hypercomputer/xpk …
505143d 
…into ppawl-use-filestore-workload
@pawloch00
cleanup
57f3428 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
Merge branch 'development' of https://github.com/AI-Hypercomputer/xpk …
86defc5 
…into ppawl-use-filestore-workload
pawloch00 and others added 17 commits November 5, 2024 12:35
@pawloch00
add new group
b05a558 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
auth docker
963a576 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
fix cluster name
7df5a06 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
fix
7ae6649 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
remove cluster removal
912dbb8 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
introduce envs with workload names
ee79e2c 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
fix newline
7273542 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
fix envs
24c2e32 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
Apply manifest file (#216)
997e594 
* create filestore instance

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* add enable filestore

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* apply manifest file

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix builds

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

---------

Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
Merge branch 'ppawl-filestore-service-account' into ppawl-use-filesto…
758cb5c 
…re-workload
@pawloch00
review fixes
fee329c 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
Merge branch 'ppawl-filestore-service-account' of https://github.com/…
9cfde52 
…AI-Hypercomputer/xpk into ppawl-filestore-service-account
@pawloch00
fix typo
2a8b19f 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
Merge branch 'ppawl-use-filestore-workload' of https://github.com/AI-…
57b7ebb 
…Hypercomputer/xpk into ppawl-use-filestore-workload
@pawloch00
fix string interpolation
77215aa 
Signed-off-by: Piotr Pawłowski <ppawl@google.com>
@pawloch00
Merge branch 'ppawl-filestore-service-account' of https://github.com/…
2f00c03 
…AI-Hypercomputer/xpk into ppawl-use-filestore-workload
@BluValor
add gcs fuse manifest example
7e57cc8
@BluValor BluValor requested a review from Obliviour as a code owner November 21, 2024 09:00
@BluValor BluValor changed the base branch from development to ppawl-use-filestore-workload November 21, 2024 09:32
Base automatically changed from ppawl-use-filestore-workload to development November 26, 2024 11:06
@BluValor BluValor force-pushed the storage-management-documentation-update branch from 0c11bce to 0d7abff Compare November 27, 2024 13:15
pawloch00
pawloch00 reviewed Nov 28, 2024
View reviewed changes
README.md Outdated Show resolved Hide resolved
@BluValor BluValor merged commit 4a988f4 into development Nov 28, 2024
7 checks passed
@BluValor BluValor deleted the storage-management-documentation-update branch November 28, 2024 13:13
pawloch00 added a commit that referenced this pull request Jan 10, 2025
@BluValor @pawloch00 @PBundyra
Storage management documentation update (#266)
355a36c 
* add flag parser and enable GCPFilestore on cluster

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* update cluster with gcp filestore

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix linter

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* filestore -> gcpfilestore

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* filetore -> gcpfilestorage

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* create filestore instance

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* add enable filestore

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* enable gcpfilestore on cluster creation

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* pyink

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* add to workload

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* pyink

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix linter

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix string formatting

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* apply manifest file

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* add workloads

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* Fix get storage function

* pyink;

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* remove adding iam

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* rm kjob

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* rm examples

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* rm duplicated

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* update filestore iam

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* cleanup

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* add readme base

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* update readme

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* README.md update

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* update Readme

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix linter

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* add storage filestore pipeline

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix builds

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix pipeline

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix naming

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* update pipeline

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix multiple addons usage

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix multiple addons usage

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix list

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* += -> append

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* comment filestore

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* add new storage

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* comment filestore

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* rm obsolete steps

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* create unique cluster and dont rm it

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* move filestore to separate job

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix networks

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* add new group

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* auth docker

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix cluster name

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* remove cluster removal

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* introduce envs with workload names

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix newline

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix envs

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* Apply manifest file (#216)

* create filestore instance

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* add enable filestore

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* apply manifest file

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix builds

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

---------

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* review fixes

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix typo

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* fix string interpolation

Signed-off-by: Piotr Pawłowski <ppawl@google.com>

* add gcs fuse manifest example

* change example namespace to default

* resolve comments

* cleanup

---------

Signed-off-by: Piotr Pawłowski <ppawl@google.com>
Co-authored-by: Piotr Pawłowski <ppawl@google.com>
Co-authored-by: Patryk Bundyra <pbundyra@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pawloch00 pawloch00 pawloch00 left review comments

@Obliviour Obliviour Awaiting requested review from Obliviour Obliviour is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@BluValor @pawloch00 @PBundyra