Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Promote Datasets to stable and clean-up unit tests #3712

Merged
Merged
Show file tree
Hide file tree
Changes from 92 commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
c65f906
Create stable datasets api from experimental
Jun 29, 2023
5dbca14
add promoted_experimental_warning_wrapper to dataset
Jun 29, 2023
4182d2e
pre-commit style fix
Jun 29, 2023
687db8e
Merge pull request #2 from nv-rliu/promote-datasets
nv-rliu Jun 29, 2023
a5348a1
Update notebooks to use stable datasets api
Jun 29, 2023
01d4e7b
Update docstrings to use stable datasets api
Jun 30, 2023
7157cb7
Update unit tests to use stable datasets API
Jun 30, 2023
1ec0ef8
Removed variants of the Karate dataset
Jun 30, 2023
0f9aa09
Moving dataset batches into testing/utils.py
Jul 3, 2023
965ec22
Expand test_dataset.py unit test coverage
Jul 3, 2023
c1281aa
Correcting YAML file inconsistencies
Jul 3, 2023
cc3973d
Update unit tests to use migrated dataset batches. Flag tests that ar…
Jul 5, 2023
1f3184b
Merge branch 'rapidsai:branch-23.08' into branch-23.08
nv-rliu Jul 5, 2023
543b770
Merge branch 'rapidsai:branch-23.08' into branch-23.08
nv-rliu Jul 6, 2023
bc27113
Merge branch 'rapidsai:branch-23.08' into branch-23.08
nv-rliu Jul 7, 2023
15a00f9
Removed outdated dataset batches from unit tests
Jul 7, 2023
b5d7ae0
Merge branch 'rapidsai:branch-23.08' into stable-datasets-notebooks
nv-rliu Jul 10, 2023
aa25d8a
Merge branch 'branch-23.08' into stable-datasets-notebooks
Jul 10, 2023
52cec16
Merge branch 'stable-datasets-notebooks' of https://github.com/nv-rli…
Jul 10, 2023
2c55538
Merge pull request #3 from nv-rliu/stable-datasets-notebooks
nv-rliu Jul 10, 2023
a7c5d1f
empty commit to re-run CI notebook checks
Jul 11, 2023
8309ce1
refactored remaining tests using experimental datasets
Jul 11, 2023
552fa2b
fix failing notebook
Jul 11, 2023
294166c
Update dataset batch naming scheme. update unit tests.
Jul 13, 2023
0e7e881
remove warnings import before nx
Jul 13, 2023
3b47cdf
Revert "remove warnings import before nx"
nv-rliu Jul 13, 2023
60bc725
Update concurrent_unordered_map.cuh
naimnv Jul 17, 2023
8119705
Fix include path for default_has
Jul 17, 2023
f182366
Fix include path for default_hash and hash_functions
Jul 17, 2023
128cadc
test_dataset.py verifies experimental DeprecationWarnings
Jul 17, 2023
80e0a00
Merge pull request #4 from naimnv/patch-1
nv-rliu Jul 17, 2023
01212d6
Update notebook headers to correct CUDA version
Jul 18, 2023
cf2d10d
Merge branch 'branch-23.08' of https://github.com/nv-rliu/cugraph int…
Jul 18, 2023
1780150
Merge branch 'rapidsai:branch-23.08' into branch-23.08
nv-rliu Jul 18, 2023
a897d45
Minor notebook changes and fixing test bug
Jul 18, 2023
b010f22
Merge branch 'branch-23.08' of https://github.com/nv-rliu/cugraph int…
Jul 18, 2023
864402d
Clear eigenvector notebook outputs
Jul 18, 2023
ab699be
cleanup unit test imports and remove nx warnings
Jul 18, 2023
cf36395
Dataset unit test comment
Jul 18, 2023
f8611c7
Merge branch 'rapidsai:branch-23.08' into branch-23.08
nv-rliu Jul 19, 2023
6658444
Fix style issue in unit test
Jul 19, 2023
15702b6
Merge branch 'branch-23.08' of https://github.com/nv-rliu/cugraph int…
Jul 19, 2023
d7026fb
Merge branch 'rapidsai:branch-23.08' into remove-nx-suppression-warnings
nv-rliu Jul 19, 2023
6b41038
Fix link_prediction import issue
Jul 19, 2023
72a6ecf
Improved DeprecationWarning check for test_dataset.py
Jul 20, 2023
0ed5a7f
Merge branch 'rapidsai:branch-23.08' into remove-nx-suppression-warnings
nv-rliu Jul 20, 2023
144a039
Merge branch 'remove-nx-suppression-warnings' into branch-23.08
nv-rliu Jul 20, 2023
baa8102
Merge pull request #5 from nv-rliu/branch-23.08
nv-rliu Jul 20, 2023
d6c216b
style fix
nv-rliu Jul 20, 2023
fb70c63
Merge pull request #6 from nv-rliu/branch-23.08
nv-rliu Jul 20, 2023
3d15c7d
more style fix
nv-rliu Jul 20, 2023
f2b3c62
Merge pull request #7 from nv-rliu/branch-23.08
nv-rliu Jul 20, 2023
f9b50a5
Migrate URLs to s3
Jul 20, 2023
b2102e8
Merge branch 'rapidsai:branch-23.08' into branch-23.08
nv-rliu Jul 20, 2023
121c888
Add re-import fixture to dataset unit test
Jul 20, 2023
6a3726a
Merge pull request #8 from nv-rliu/branch-23.08
nv-rliu Jul 20, 2023
c8aa44f
Update Ktruss polbooks metadata URL
Jul 21, 2023
df6044e
Merge pull request #9 from nv-rliu/branch-23.08
nv-rliu Jul 21, 2023
02e23ef
Commit to see if #3680 updates
Jul 21, 2023
e8ead8e
Merge branch 'rapidsai:branch-23.08' into remove-nx-suppression-warnings
nv-rliu Jul 24, 2023
eb47478
Update Datasets with standardized 'download' parameter
Jul 24, 2023
7d66b36
Update doc strings: fetch -> Download
Jul 24, 2023
e5e9b71
Update unit test usage of datasets API: fetch -> download
Jul 24, 2023
a3bb2b3
Cleanup & standardize metadata files
Jul 24, 2023
30734f9
Merge branch 'rapidsai:branch-23.08' into branch-23.08
nv-rliu Jul 24, 2023
9d96209
Merge pull request #10 from nv-rliu/branch-23.08
nv-rliu Jul 24, 2023
820dc51
Update metadata fields
Jul 24, 2023
56d3682
Merge branch 'remove-nx-suppression-warnings' of https://github.com/n…
Jul 24, 2023
ccbae16
Style fix. test_dataset.py WIP
Jul 24, 2023
1f30d2a
Merge branch 'branch-23.08' into remove-nx-suppression-warnings
nv-rliu Jul 24, 2023
3073ec0
Update yaml
Jul 25, 2023
0768c95
Merge branch 'branch-23.08' into remove-nx-suppression-warnings
nv-rliu Jul 25, 2023
5dfb2f2
Merge branch 'remove-nx-suppression-warnings' of https://github.com/n…
Jul 25, 2023
df9b19c
Fix experimental packages' docstrings 'fetch' -> 'download'
Jul 25, 2023
288b840
Merge branch 'branch-23.08' into remove-nx-suppression-warnings
nv-rliu Jul 25, 2023
53ddb16
Notebook algorithm fix
Jul 25, 2023
027de9a
Add unit test to verify self-loops
Jul 25, 2023
c1c2de9
Merge branch 'branch-23.08' into branch-23.08
nv-rliu Jul 25, 2023
51a1089
Merge pull request #11 from nv-rliu/branch-23.08
nv-rliu Jul 25, 2023
fc485dc
Merge branch 'rapidsai:branch-23.08' into remove-nx-suppression-warnings
nv-rliu Jul 25, 2023
4cfca07
Update notebooks: 'fetch' -> 'download'
Jul 25, 2023
336f57c
Update naming scheme with all underscores
Jul 25, 2023
b89b363
Unable to change 'name' field to adhere to underscore naming conventi…
Jul 26, 2023
1e7cb63
Merge branch 'branch-23.08' into remove-nx-suppression-warnings
nv-rliu Jul 26, 2023
c519590
Update Sorensen notebook to use stable
Jul 26, 2023
958af97
Fix bulk sampling style
Jul 26, 2023
fb9e75e
Remove ktruss_polbooks from cugraph.datasets and refactor test_k_trus…
Jul 26, 2023
f173e13
Add unit test to check for graph symmetry and update metadata YAML files
Jul 26, 2023
87dd183
Update metadata files based on feedback from @rlratzel
Jul 27, 2023
c699844
Merge branch 'branch-23.08' into remove-nx-suppression-warnings
nv-rliu Jul 27, 2023
e4754c6
Update github workflow pr.yaml and build.yaml
Jul 27, 2023
05182ef
Add missing netscience import to test_batch_edge_betweenness_centrali…
Jul 27, 2023
1fa0496
Fix YAML file formatting issue
Jul 27, 2023
77d0cd3
Merge branch 'rapidsai:branch-23.08' into remove-nx-suppression-warnings
nv-rliu Jul 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ concurrency:

jobs:
cpp-build:
node_type: cpu32
secrets: inherit
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-build.yaml@branch-23.08
with:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
uses: rapidsai/shared-action-workflows/.github/workflows/conda-cpp-build.yaml@branch-23.08
with:
build_type: pull-request
node_type: cpu16
node_type: cpu32
conda-cpp-tests:
needs: conda-cpp-build
secrets: inherit
Expand Down
25 changes: 20 additions & 5 deletions notebooks/algorithms/centrality/Betweenness.ipynb
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -13,10 +14,12 @@
"| Brad Rees | 04/24/2019 | created | 0.15 | GV100, CUDA 11.0\n",
"| Brad Rees | 08/16/2020 | tested / updated | 21.10 nightly | RTX 3090 CUDA 11.4\n",
"| Don Acosta | 07/05/2022 | tested / updated | 22.08 nightly | DGX Tesla V100 CUDA 11.5\n",
"| Ralph Liu | 07/26/2022 | updated | 22.08 nightly | DGX Tesla V100 CUDA 11.5"
"| Ralph Liu | 07/26/2022 | updated | 22.08 nightly | DGX Tesla V100 CUDA 11.5\n",
"| | 06/29/2023 | updated | 23.08 nightly | DGX Tesla V100 CUDA 12.0"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -28,6 +31,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -37,6 +41,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand Down Expand Up @@ -71,6 +76,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -82,6 +88,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -98,6 +105,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -115,7 +123,7 @@
"import cudf\n",
"\n",
"# Import a built-in dataset\n",
"from cugraph.experimental.datasets import karate"
"from cugraph.datasets import karate"
]
},
{
Expand All @@ -129,6 +137,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -142,10 +151,11 @@
"outputs": [],
"source": [
"# Create a graph using the imported Dataset object\n",
"G = karate.get_graph(fetch=True)"
"G = karate.get_graph(download=True)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand Down Expand Up @@ -173,6 +183,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand Down Expand Up @@ -203,13 +214,15 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"---"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand Down Expand Up @@ -267,18 +280,20 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"As mentioned, the scores are different but the ranking is the same."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"___\n",
"Copyright (c) 2019-2022, NVIDIA CORPORATION.\n",
"Copyright (c) 2019-2023, NVIDIA CORPORATION.\n",
"\n",
"Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0\n",
"\n",
Expand All @@ -303,7 +318,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.7"
"version": "3.10.12"
},
"vscode": {
"interpreter": {
Expand Down
23 changes: 20 additions & 3 deletions notebooks/algorithms/centrality/Centrality.ipynb
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Centrality"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -18,10 +20,12 @@
"| Brad Rees | 04/16/2021 | created | 0.19 | GV100, CUDA 11.0\n",
"| Brad Rees | 08/05/2021 | tested / updated | 21.10 nightly | RTX 3090 CUDA 11.4\n",
"| Don Acosta | 07/29/2022 | tested / updated | 22.08 nightly | DGX Tesla V100 CUDA 11.5\n",
"| Ralph Liu | 06/29/2023 | updated | 23.08 nightly | DGX Tesla V100 CUDA 12.0\n",
" "
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand Down Expand Up @@ -86,13 +90,15 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"---"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -116,7 +122,7 @@
"# Import the cugraph modules\n",
"import cugraph\n",
"import cudf\n",
"from cugraph.experimental.datasets import karate"
"from cugraph.datasets import karate"
]
},
{
Expand All @@ -132,6 +138,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand Down Expand Up @@ -194,6 +201,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -203,6 +211,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -216,10 +225,11 @@
"outputs": [],
"source": [
"# Create a graph using the imported Dataset object\n",
"G = karate.get_graph(fetch=True)"
"G = karate.get_graph(download=True)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -236,6 +246,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -254,6 +265,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand Down Expand Up @@ -312,6 +324,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -320,6 +333,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand Down Expand Up @@ -369,6 +383,7 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand All @@ -385,13 +400,15 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"There are differences in how each centrality measure ranks the nodes. In some cases, every algorithm returns similar results, and in others, the results are different. Understanding how the centrality measure is computed and what edge represent is key to selecting the right centrality metric."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
Expand Down Expand Up @@ -420,7 +437,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.15"
"version": "3.10.12"
},
"vscode": {
"interpreter": {
Expand Down
Loading
Loading