Skip to content

Commit

Permalink
Update datasets download URL
Browse files Browse the repository at this point in the history
Signed-off-by: Jordan Jacobelli <jjacobelli@nvidia.com>
  • Loading branch information
jjacobelli committed Feb 28, 2023
1 parent 08ae65a commit 96829b4
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 11 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ cpp/thirdparty/googletest/
## Datasets
datasets/*
!datasets/cyber.csv
!datasets/get_test_data.sh
!datasets/karate-data.csv
!datasets/karate_undirected.csv
!datasets/karate-disjoint.csv
Expand Down
22 changes: 11 additions & 11 deletions datasets/get_test_data.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,45 +26,45 @@ set -o pipefail
# confusing now that there's dedicated datasets for benchmarks.
BASE_DATASET_DATA="
# ~22s download
https://s3.us-east-2.amazonaws.com/rapidsai-data/cugraph/test/datasets.tgz
https://data.rapids.ai/cugraph/test/datasets.tgz
test
# ~14s download
https://s3.us-east-2.amazonaws.com/rapidsai-data/cugraph/test/ref/pagerank.tgz
https://data.rapids.ai/cugraph/test/ref/pagerank.tgz
test/ref
# ~1s download
https://s3.us-east-2.amazonaws.com/rapidsai-data/cugraph/test/ref/sssp.tgz
https://data.rapids.ai/cugraph/test/ref/sssp.tgz
test/ref
# ~15s download
https://s3.us-east-2.amazonaws.com/rapidsai-data/cugraph/benchmark/hibench/hibench_1_large.tgz
https://data.rapids.ai/cugraph/benchmark/hibench/hibench_1_large.tgz
benchmark
# ~1s download
https://s3.us-east-2.amazonaws.com/rapidsai-data/cugraph/benchmark/hibench/hibench_1_small.tgz
https://data.rapids.ai/cugraph/benchmark/hibench/hibench_1_small.tgz
benchmark
# ~0.6s download
https://rapidsai-data.s3.us-east-2.amazonaws.com/cugraph/test/tsplib/datasets.tar.gz
https://data.rapids.ai/cugraph/test/tsplib/datasets.tar.gz
tsplib
"

EXTENDED_DATASET_DATA="
# ~42s download - tests using this dataset are currently not run in test.sh with --quick
https://s3.us-east-2.amazonaws.com/rapidsai-data/cugraph/benchmark/hibench/hibench_1_huge.tgz
https://data.rapids.ai/cugraph/benchmark/hibench/hibench_1_huge.tgz
benchmark
"

BENCHMARK_DATASET_DATA="
# ~90s download - these are used for benchmarks runs (code in <cugraph root>/benchmarks)
https://rapidsai-data.s3.us-east-2.amazonaws.com/cugraph/benchmark/benchmark_csv_data.tgz
https://data.rapids.ai/cugraph/benchmark/benchmark_csv_data.tgz
csv
"

SELF_LOOPS_DATASET_DATA="
# ~1s download
https://rapidsai-data.s3.us-east-2.amazonaws.com/cugraph/benchmark/benchmark_csv_data_self_loops.tgz
# ~1s download
https://data.rapids.ai/cugraph/benchmark/benchmark_csv_data_self_loops.tgz
self_loops
"
################################################################################
Expand All @@ -86,7 +86,7 @@ if hasArg "--benchmark"; then
DATASET_DATA="${BENCHMARK_DATASET_DATA}"
elif hasArg "--subset"; then
DATASET_DATA="${BASE_DATASET_DATA}"
elif hasArg "--self_loops"; then
elif hasArg "--self_loops"; then
DATASET_DATA="${SELF_LOOPS_DATASET_DATA}"
# Do not include benchmark datasets by default - too big
else
Expand Down

0 comments on commit 96829b4

Please sign in to comment.