Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add distributed APIs into doc #23

Merged
merged 2 commits into from
Jun 12, 2022
Merged

Add distributed APIs into doc #23

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cab4722
Add distributed data-load and query-load command
NLGithubWP May 29, 2022
c34dd5f
Merge branch 'main' of github.com:COOL-cohort/COOL-site
NLGithubWP May 29, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 23 additions & 1 deletion docs/getting-started/distributed-deployment.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ Update hdfs.host, zookeeper.host, and server host

### Datasets

Manually upload used Cublet, query.json file to HDFS
Manually upload used Cublet, query.json file to HDFS, or use the following APIs to upload data, table.yaml, and query to HDFS.

- Upload all partitioned .dz files to HDFS path: `/cube/` eg. `"/cube/health/v1/1805b2fdb75v2.dz"`, `"/cube/health/v1/1804bc18968.dz"`
- Upload query.json file to `/tmp/queryID` folder, eg. `"/tmp/1/query.json"`
Expand All @@ -66,6 +66,28 @@ Manually upload used Cublet, query.json file to HDFS

In distributed mode, the client can only talk to the broker.

- \[server:port]: broker/load-data-to-hdfs

Upload data and table into HDFS for future usage.

This API requires the CSV and table.YAML files are already on the server-side.

```shell
curl --location --request POST 'http://127.0.0.1:9013/broker/load-data-to-hdfs' \
--header 'Content-Type: application/json' \
--data-raw '{"dataFileType": "CSV", "cubeName": "health", "schemaPath": "health/table.yaml", "dimPath": "health/dim.csv", "dataPath": "health/raw2.csv", "outputPath": "datasetSource"}'
```

- \[server:port]: broker/load-query-to-hdfs

Upload the query file into HDFS for the future query.

```shell
curl --location --request POST 'http://127.0.0.1:9013/broker/load-query-to-hdfs' \
--header 'Content-Type: multipart/form-data' \
--form 'queryFile=@"FULL_PATH_PREFIX/COOL/health/query.json"'
```

- \[server:port]: info

- List all workable urls
Expand Down