Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Subtask] Add web UI support for fileset catalog #2292

Closed
Tracked by #1241
jerryshao opened this issue Feb 21, 2024 · 3 comments · Fixed by #2721
Closed
Tracked by #1241

[Subtask] Add web UI support for fileset catalog #2292

jerryshao opened this issue Feb 21, 2024 · 3 comments · Fixed by #2721
Assignees
@LauraXia123
Labels
subtask Subtasks of umbrella issue
Milestone
Gravitino 0.5.0

Comments

@jerryshao
Copy link
Contributor

jerryshao commented Feb 21, 2024 •
edited
Loading

Describe the subtask

This issue tracks the work of add UI for fileset catalog. User can view the created filesets on the web UI.

Parent issue

#1241
@jerryshao jerryshao added the subtask Subtasks of umbrella issue label Feb 21, 2024
@jerryshao jerryshao added this to the Gravitino 0.5.0 milestone Feb 21, 2024
@jerryshao jerryshao mentioned this issue Feb 21, 2024
Closed
18 tasks
@coolderli
Copy link
Collaborator

coolderli commented Feb 22, 2024 •
edited
Loading

@jerryshao Do we need some operations like UploadFiles, DownloadFiles, PreviewFile, and ListFiles?

  • UploadFiles: There may some users who want to upload local files to a fileset.
  • DownloadFiles: I'm not sure, maybe users want to open one file to test.
  • PreviewFile: open a file. Some of my users like to open a file such as CSV on web UI, and take a look at the content.
  • ListFiles: Some of our users like to look at the directory to check if the commit meets expectations. And see how many files or directories are in the specified path.
  • DeleteFiles: delete some files manually.

@jerryshao
Copy link
Contributor Author

IMO, I would mainly focus on the read-only/visualization functions in this version. Things you mentioned are valuable but not so necessary, this can be achieved in different ways. Currently, we're lacking of UI person, so I would think of having a basic functionality first.

@coolderli
Copy link
Collaborator

IMO, I would mainly focus on the read-only/visualization functions in this version. Things you mentioned are valuable but not so necessary, this can be achieved in different ways. Currently, we're lacking of UI person, so I would think of having a basic functionality first.

Got it. Thanks.

@LauraXia123 LauraXia123 self-assigned this Mar 20, 2024
LauraXia123 added a commit to LauraXia123/gravitino that referenced this issue Mar 28, 2024
@LauraXia123
[apache#2292] add web ui support for fileset catalog
a08636b
@LauraXia123 LauraXia123 mentioned this issue Mar 28, 2024
Merged
LauraXia123 added a commit to LauraXia123/gravitino that referenced this issue Mar 28, 2024
@LauraXia123
[apache#2292] add web ui support for fileset catalog
4fdedc3
ch3yne added a commit that referenced this issue Apr 1, 2024
@LauraXia123 @ch3yne
[#2292]feat(fileset): add web ui support for fileset catalog (#2721)
db36332 
### What changes were proposed in this pull request?

Web ui support for fileset catalog
Some try catch which helpful for debugging among extraReducers functions

### Why are the changes needed?
<img width="1555" alt="image"
src="https://github.com/datastrato/gravitino/assets/9210625/118ac5f9-c1aa-4fc5-b3fe-496cac32bcef">

Fix: #2292

### Does this PR introduce _any_ user-facing change?
N/A

### How was this patch tested?
run catalog e2e test successfully

![catalog_e2e_test](https://github.com/datastrato/gravitino/assets/9210625/6933cbc0-9ed2-44c4-ae09-a9dfcc05e7b2)

---------

Co-authored-by: CHEYNE <contact@ch3yne.com>
LauraXia123 added a commit to LauraXia123/gravitino that referenced this issue Apr 1, 2024
@LauraXia123
[apache#2292] add web ui support for fileset catalog
41bd74a
xunliu added a commit to xunliu/gravitino that referenced this issue Apr 17, 2024
@xunliu
[apache#2292] improvement(PyClient): Fix unstable Python client integ…
13f8897 
…ration test in Github Action
@xunliu xunliu mentioned this issue Apr 17, 2024
Merged
xunliu added a commit to xunliu/gravitino that referenced this issue Apr 17, 2024
@xunliu
[apache#2292] improvement(PyClient): Fix unstable Python client integ…
d71aeb4 
…ration test in Github Action
xunliu added a commit to xunliu/gravitino that referenced this issue Apr 18, 2024
@xunliu
[apache#2292] improvement(PyClient): Fix unstable Python client integ…
bf117e8 
…ration test in Github Action
xunliu added a commit to xunliu/gravitino that referenced this issue Apr 18, 2024
@xunliu
[apache#2292] improvement(PyClient): Fix unstable Python client integ…
32e39e5 
…ration test
xunliu added a commit to xunliu/gravitino that referenced this issue Apr 18, 2024
@xunliu
[apache#2292] improvement(PyClient): Fix unstable Python client integ…
16ef973 
…ration test
jerryshao pushed a commit that referenced this issue Apr 18, 2024
@xunliu
[#2292] improvement(PyClient): Fix unstable Python client integration…
9e6e189 
… test (#2994)

### What changes were proposed in this pull request?

1. Add `skipPyClientITs` param in Gradle build script. Let's Python
client integration test not running in the backend ITs
2. Combined Gradle build command of Python client test and integration
test
3. improvement FilesetCatalog integration test codes.
4. Test mode

+ Test Principle: Every Python ITs class base on the
`IntegrationTestEnv`, `IntegrationTestEnv` will automatic start and stop
Gravitino server to support ITs, But when you run multiple ITs class at
same time, The first test class that finishes running will shut down the
Gravitino server, which will cause other test classes to fail if they
can't connect to the Gravitino server.
+ Run test in the IDE: Through `IntegrationTestEnv` class automatic
start and stop Gravitino server to support ITs.
+ Run test in the Github Action: Manual start and stop Gravitino server,
and set `EXTERNAL_START_GRAVITINO` environment variable
+ Run test in the Gradle command `:client:client-python:test`: Gradle
automatic start and stop Gravitino server, and set
`EXTERNAL_START_GRAVITINO` environment variable.

### Why are the changes needed?

Fix: #2292

### Does this PR introduce _any_ user-facing change?

N/A

### How was this patch tested?

Through CI check.
diqiu50 pushed a commit to diqiu50/gravitino that referenced this issue Jun 13, 2024
@xunliu @diqiu50
[apache#2292] improvement(PyClient): Fix unstable Python client integ…
0610bcf 
…ration test (apache#2994)

### What changes were proposed in this pull request?

1. Add `skipPyClientITs` param in Gradle build script. Let's Python
client integration test not running in the backend ITs
2. Combined Gradle build command of Python client test and integration
test
3. improvement FilesetCatalog integration test codes.
4. Test mode

+ Test Principle: Every Python ITs class base on the
`IntegrationTestEnv`, `IntegrationTestEnv` will automatic start and stop
Gravitino server to support ITs, But when you run multiple ITs class at
same time, The first test class that finishes running will shut down the
Gravitino server, which will cause other test classes to fail if they
can't connect to the Gravitino server.
+ Run test in the IDE: Through `IntegrationTestEnv` class automatic
start and stop Gravitino server to support ITs.
+ Run test in the Github Action: Manual start and stop Gravitino server,
and set `EXTERNAL_START_GRAVITINO` environment variable
+ Run test in the Gradle command `:client:client-python:test`: Gradle
automatic start and stop Gravitino server, and set
`EXTERNAL_START_GRAVITINO` environment variable.

### Why are the changes needed?

Fix: apache#2292

### Does this PR introduce _any_ user-facing change?

N/A

### How was this patch tested?

Through CI check.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@LauraXia123 LauraXia123

Labels
subtask Subtasks of umbrella issue
Projects
None yet
Milestone
Gravitino 0.5.0
Development

Successfully merging a pull request may close this issue.

[#2292]feat(fileset): add web ui support for fileset catalog LauraXia123/gravitino
3 participants
@jerryshao @LauraXia123 @coolderli