Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add sample for revoking dataset access #778

Merged
merged 53 commits into from
Dec 6, 2021
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
0abac4c
revoke dataset access setup
Jul 1, 2021
d699bec
basic template for sample
Jul 2, 2021
98b245d
sample + test
Jul 7, 2021
f1adee7
revoke dataset access sample
Jul 16, 2021
e729bbd
🦉 Updates from OwlBot
gcf-owl-bot[bot] Jul 16, 2021
b80c4bf
docs: add sample for revoking dataset access - update year and string…
Jul 23, 2021
2960b48
Merge branch 'bigquery_remove_view_access' of github.com:googleapis/p…
Jul 23, 2021
02bd2b4
docs: add sample for revoking dataset access - move to snippets and c…
Jul 23, 2021
fa0e48c
moving update_dataset to /snippets and adjusting imports on both revo…
Jul 27, 2021
98bb4a2
Update samples/snippets/revoke_dataset_access.py
loferris Jul 27, 2021
8c4bc7a
Update samples/snippets/revoke_dataset_access.py
loferris Jul 27, 2021
5197d0f
🦉 Updates from OwlBot
gcf-owl-bot[bot] Jul 27, 2021
cfb3805
updated test
Jul 29, 2021
4be9760
🦉 Updates from OwlBot
gcf-owl-bot[bot] Jul 29, 2021
0a57eba
Merge branch 'main' of github.com:googleapis/python-bigquery into big…
Aug 27, 2021
d4672f2
Merge branch 'main' into bigquery_remove_view_access
loferris Aug 30, 2021
b9e35aa
change after running test
Aug 30, 2021
b076972
Merge branch 'bigquery_remove_view_access' of github.com:googleapis/p…
Aug 30, 2021
4ec3512
Merge branch 'main' into bigquery_remove_view_access
tswast Sep 10, 2021
15d84b1
Merge branch 'main' into bigquery_remove_view_access
loferris Sep 11, 2021
2209d8e
resolving linting failure, rewriting test
Sep 11, 2021
0da2301
removed relative import errors
Sep 15, 2021
77066a5
remove relative mport from update_dataset_access
Sep 15, 2021
32d9b71
adding fixture to conftest.py
Sep 15, 2021
b7e91eb
Merge branch 'main' into bigquery_remove_view_access
plamut Sep 17, 2021
318242d
Merge branch 'main' into bigquery_remove_view_access
tswast Oct 6, 2021
5fe08d0
Merge branch 'main' into bigquery_remove_view_access
parthea Oct 6, 2021
1c4469b
Merge branch 'main' into bigquery_remove_view_access
meredithslota Oct 13, 2021
fe66df2
Merge branch 'main' into bigquery_remove_view_access
loferris Nov 18, 2021
fbcc09f
updated sample
Nov 18, 2021
4234450
updating sample to match new update_access sample
Nov 19, 2021
2b82c45
Merge branch 'main' into bigquery_remove_view_access
loferris Nov 19, 2021
2ea0f4e
fixing region tags
Nov 19, 2021
6bbdc33
consolidated tests into one file for both methods
Nov 19, 2021
a493adc
Merge branch 'bigquery_remove_view_access' of github.com:googleapis/p…
Nov 19, 2021
575260a
updating test to full_dataset format
Nov 19, 2021
27d8170
updated revoke sample
Nov 20, 2021
adc1c76
updating test
Nov 20, 2021
62ec25e
refactored sample
Nov 22, 2021
072b785
Update samples/snippets/conftest.py
tswast Nov 23, 2021
dade3b8
Update samples/snippets/revoke_dataset_access.py
loferris Nov 23, 2021
c74022e
Update samples/snippets/update_dataset_access.py
loferris Nov 23, 2021
9da6de6
Update samples/snippets/revoke_dataset_access.py
loferris Nov 23, 2021
ede9158
Update samples/snippets/revoke_dataset_access.py
loferris Nov 23, 2021
05a4f19
refactoring entry
Nov 24, 2021
6c04f28
added comment for entry access
Nov 24, 2021
d030e21
Merge branch 'main' into bigquery_remove_view_access
tswast Nov 29, 2021
023eb96
Update samples/snippets/README.rst
loferris Nov 29, 2021
6d9f274
Update samples/snippets/dataset_access_test.py
loferris Nov 29, 2021
3fd6856
Update samples/snippets/dataset_access_test.py
loferris Nov 29, 2021
6ce391d
added develper TODO in sample
Nov 29, 2021
04497a1
add comments to samples
tswast Dec 6, 2021
2e98eca
Merge branch 'main' into bigquery_remove_view_access
tswast Dec 6, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 6 additions & 23 deletions samples/snippets/README.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

.. This file is automatically generated. Do not edit this file directly.

Google BigQuery Python Samples
Expand All @@ -16,11 +15,14 @@ This directory contains samples for Google BigQuery. `Google BigQuery`_ is Googl
.. _Google BigQuery: https://cloud.google.com/bigquery/docs


To run the sample, you need to have `BigQuery Admin` role.
loferris marked this conversation as resolved.
Show resolved Hide resolved



Setup
-------------------------------------------------------------------------------



Authentication
++++++++++++++

Expand All @@ -31,9 +33,6 @@ credentials for applications.
.. _Authentication Getting Started Guide:
https://cloud.google.com/docs/authentication/getting-started




Install Dependencies
++++++++++++++++++++

Expand All @@ -48,7 +47,7 @@ Install Dependencies
.. _Python Development Environment Setup Guide:
https://cloud.google.com/python/setup

#. Create a virtualenv. Samples are compatible with Python 3.6+.
#. Create a virtualenv. Samples are compatible with Python 2.7 and 3.4+.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@parthea Looks like this is a reversion by owlbot.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first glance, this change wasn't made by owlbot. See cfb3805#diff-80e5f0b5c748e24702257bf661911a6c240d81c82a5029056391211591a4a5c9

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@loferris Please revert the changes to this file.


.. code-block:: bash

Expand All @@ -64,15 +63,9 @@ Install Dependencies
.. _pip: https://pip.pypa.io/
.. _virtualenv: https://virtualenv.pypa.io/






Samples
-------------------------------------------------------------------------------


Quickstart
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Expand All @@ -89,8 +82,6 @@ To run this sample:
$ python quickstart.py




Simple Application
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Expand All @@ -107,8 +98,6 @@ To run this sample:
$ python simple_app.py




User Credentials
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Expand All @@ -124,7 +113,6 @@ To run this sample:

$ python user_credentials.py


usage: user_credentials.py [-h] [--launch-browser] project

Command-line application to run a query using user credentials.
Expand All @@ -143,10 +131,6 @@ To run this sample:







The client library
-------------------------------------------------------------------------------

Expand All @@ -162,5 +146,4 @@ to `browse the source`_ and `report issues`_.
https://github.com/GoogleCloudPlatform/google-cloud-python/issues



.. _Google Cloud SDK: https://cloud.google.com/sdk/
.. _Google Cloud SDK: https://cloud.google.com/sdk/
5 changes: 5 additions & 0 deletions samples/snippets/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,11 @@ def dataset_id(bigquery_client: bigquery.Client, project_id: str):
bigquery_client.delete_dataset(dataset, delete_contents=True, not_found_ok=True)


@pytest.fixture(scope="session")
def entity_id(bigquery_client: bigquery.Client, dataset_id: str):
return "cloud-developer-relations@google.com"


@pytest.fixture(scope="session")
def dataset_id_us_east1(bigquery_client: bigquery.Client, project_id: str):
dataset_id = prefixer.create_prefix()
Expand Down
49 changes: 49 additions & 0 deletions samples/snippets/dataset_access_permissions_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Copyright 2021 Google LLC
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: In the filename, "access" and "permissions" are redundant. Let's call it dataset_access_test.py.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in latest commit!

#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import revoke_dataset_access
import update_dataset_access


def test_dataset_access_permissions(capsys, dataset_id, entity_id, bigquery_client):
original_dataset = bigquery_client.get_dataset(dataset_id)
update_dataset_access.update_dataset_access(dataset_id)
full_dataset_id = "{}.{}".format(
original_dataset.project, original_dataset.dataset_id
)

out, err = capsys.readouterr()
assert (
"Updated dataset '{}' with modified user permissions.".format(full_dataset_id)
in out
)

updated_dataset = bigquery_client.get_dataset(dataset_id)
updated_dataset_entries = list(updated_dataset.access_entries)
revoke_dataset_access.revoke_dataset_access(dataset_id, entity_id)
revoked_dataset = bigquery_client.get_dataset(dataset_id)
revoked_dataset_entries = list(revoked_dataset.access_entries)

full_dataset_id = f"{updated_dataset.project}.{updated_dataset.dataset_id}"
out, err = capsys.readouterr()
assert (
f"Revoked dataset access for '{entity_id}' to ' dataset '{full_dataset_id}.'"
in out
)
assert len(revoked_dataset_entries) == len(updated_dataset_entries) - 1
is_revoked = 0
for entry in revoked_dataset_entries:
if entry.entity_id == entity_id:
is_revoked += 1
assert is_revoked == 0
54 changes: 54 additions & 0 deletions samples/snippets/revoke_dataset_access.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.


def revoke_dataset_access(dataset_id, entity_id):
tswast marked this conversation as resolved.
Show resolved Hide resolved
loferris marked this conversation as resolved.
Show resolved Hide resolved
# [START bigquery_revoke_dataset_access]

# TODO(developer): Set dataset_id to the ID of the dataset to fetch.
# dataset_id = 'your-project.your_dataset'

# TODO(developer): Set entity_id to the ID of the email or group from whom you are revoking access.
# entity_id = "user-or-group-to-remove@example.com"
loferris marked this conversation as resolved.
Show resolved Hide resolved

from google.cloud import bigquery

# Construct a BigQuery client object.
client = bigquery.Client()

dataset = client.get_dataset(dataset_id) # Make an API request.

entry = bigquery.AccessEntry(
role="READER",
entity_type="groupByEmail",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a comment saying that the developer will need to set this to the correct type.

Although, perhaps we can delete this whole line 32-36? A code sample should only contain the logic necessary for the developer to understand the concept we're trying to teach them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took it out!

entity_id="cloud-developer-relations@google.com",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be entity_id=entity_id?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me!

)

entries = list(dataset.access_entries)
entries.append(entry)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused as to why we'd add the entry and then remove it on line 42. Perhaps a copy-paste error?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's a redundancy - I was adding a specific entry to alter but I think that'll be covered in the test case and sort of muddies the water in the sample itself. I'll refactor.

dataset.access_entries = entries
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line confuses me. It gets overwritten in the next line of code, so looks unnecessary. Possibly leftover from a debugging?


dataset.access_entries = list(
filter(lambda entry: entry.entity_id != entity_id, entries)
)
loferris marked this conversation as resolved.
Show resolved Hide resolved

dataset = client.update_dataset(
dataset,
# Update just the `access_entries` property of the dataset.
["access_entries"],
) # Make an API request.

full_dataset_id = f"{dataset.project}.{dataset.dataset_id}"
print(f"Revoked dataset access for '{entity_id}' to ' dataset '{full_dataset_id}.'")
# [END bigquery_revoke_dataset_access]
45 changes: 45 additions & 0 deletions samples/snippets/update_dataset_access.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.


def update_dataset_access(dataset_id):
loferris marked this conversation as resolved.
Show resolved Hide resolved

# [START bigquery_update_dataset_access]
from google.cloud import bigquery

# Construct a BigQuery client object.
client = bigquery.Client()

# TODO(developer): Set dataset_id to the ID of the dataset to fetch.
# dataset_id = 'your-project.your_dataset'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise here, let's use the "save the original, then overwrite it" pattern.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we missed this one.


dataset = client.get_dataset(dataset_id) # Make an API request.

entry = bigquery.AccessEntry(
role="READER",
entity_type="groupByEmail",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a comment explaining the various options available, or at least a link to the docs where these are described.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added!

entity_id="cloud-developer-relations@google.com",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to use the entity_id fixture here? If so, we should add a variable above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added, changed test as well!

)

entries = list(dataset.access_entries)
entries.append(entry)
dataset.access_entries = entries

dataset = client.update_dataset(dataset, ["access_entries"]) # Make an API request.

full_dataset_id = "{}.{}".format(dataset.project, dataset.dataset_id)
print(
"Updated dataset '{}' with modified user permissions.".format(full_dataset_id)
)
# [END bigquery_update_dataset_access]