From 900a8c2c43f2648c02f56f43e8bcc42dfd8b2d73 Mon Sep 17 00:00:00 2001 From: arithmetic1728 <58957152+arithmetic1728@users.noreply.github.com> Date: Mon, 31 Aug 2020 16:22:15 -0700 Subject: [PATCH] docs: add samples from python-docs-samples/tasks (#36) * Add samples for Cloud Tasks [(#1068)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1068) * Add samples for Cloud Tasks * Respond to tasks sample review * Update app engine queues samples * Address review feedback * Address review issues and convert pull queue sample to not use API key auth * Reform pull queues to match appengine queues changes to auth, command line input, readme * flake8 and fix comment * Fix Tasks sample test issues. * Remove queue.yaml, now unused * Add required flag on certain arguments to command-line tools * Auto-update dependencies. [(#1116)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1116) * Auto-update dependencies. [(#1133)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1133) * Auto-update dependencies. * Fix missing http library Change-Id: I99faa600f2f3f1f50f57694fc9835d7f35bda250 * Update Task Queue samples for beta * Use full import to get client * Auto-update dependencies. [(#1186)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1186) * Auto-update dependencies. [(#1217)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1217) * Added "Open in Cloud Shell" buttons to README files [(#1254)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1254) * Add comments and region tags to Cloud Tasks samples [(#1271)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1271) * Standardize on CamelCase, reword confusing endpoint name [(#1288)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1288) * Auto-update dependencies. [(#1309)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1309) * Rename pull to lease and fix name/parent confusion [(#1311)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1311) * Updated region tags for pull queue samples [(#1329)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1329) * Auto-update dependencies. [(#1320)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1320) * Auto-update dependencies. [(#1355)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1355) * Auto-update dependencies. [(#1359)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1359) * Auto-update dependencies. * Update Cloud Tasks Samples [(#1529)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1529) * passing create task * Passing tests * updates to region tags * update region tags [(#1532)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1532) * update Tasks Sample for App Engine [(#1541)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1541) * update gcloud command for creating queues * deploys and runs * update license * passing tests * Fix run command [(#1563)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1563) * Updated library for TTS GA [(#1552)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1552) * update gcloud command for Cloud Tasks [(#1566)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1566) * update gcloud command * update pull queue command * update pull queue command * Update Cloud Tasks Push Queue Sample [(#1698)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1698) * deleted pull queues * updated samples * fix dependency versions * [Cloud Tasks] Move samples to new folder [(#2114)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/2114) * Move samples to keep consistent with other langauges * Ad system tests as well * [Cloud Tasks] Add task with authentication sample [(#2113)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/2113) * Add task with authentication sample * Fix linting * Fix linting * Fix spacing * Update tests with service account * Move samples and update READMEs * Update version and linting * Update task sample comments [(#2156)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/2156) * Update task comments * Update readme * Update queue name * update gcloud [(#2208)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/2208) * Add protobuf dep and install instructions [(#2250)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/2250) * Add Migration Guide Snippets for Cloud Tasks [(#2316)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/2316) * Migration guide * remove app and update migraitonn * snippets for migration guide - tests added * lint * remove print statements * Styling changes * Travis trigger * Update create_http_task.py [(#2187)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/2187) Updates `create_http_task.py` to have missing `in_seconds` variable * Adds updates for samples profiler ... vision [(#2439)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/2439) * Update Cloud Tasks library version [(#2516)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/2516) * Update to new library * update library version * Adds Task name while creating Task [(#2543)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/2543) * Task Name in Creating HTTP Task with Token [(#2700)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/2700) * chore(deps): update dependency google-cloud-tasks to v1.5.0 [(#3168)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/3168) * chore(deps): update dependency googleapis-common-protos to v1.51.0 [(#3171)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/3171) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [googleapis-common-protos](https://togithub.com/googleapis/googleapis) | minor | `==1.6.0` -> `==1.51.0` | --- ### Renovate configuration :date: **Schedule**: At any time (no schedule defined). :vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied. :recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox. :no_bell: **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples). * Simplify noxfile setup. [(#2806)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/2806) * chore(deps): update dependency requests to v2.23.0 * Simplify noxfile and add version control. * Configure appengine/standard to only test Python 2.7. * Update Kokokro configs to match noxfile. * Add requirements-test to each folder. * Remove Py2 versions from everything execept appengine/standard. * Remove conftest.py. * Remove appengine/standard/conftest.py * Remove 'no-sucess-flaky-report' from pytest.ini. * Add GAE SDK back to appengine/standard tests. * Fix typo. * Roll pytest to python 2 version. * Add a bunch of testing requirements. * Remove typo. * Add appengine lib directory back in. * Add some additional requirements. * Fix issue with flake8 args. * Even more requirements. * Readd appengine conftest.py. * Add a few more requirements. * Even more Appengine requirements. * Add webtest for appengine/standard/mailgun. * Add some additional requirements. * Add workaround for issue with mailjet-rest. * Add responses for appengine/standard/mailjet. Co-authored-by: Renovate Bot * Remove dependency googleapis-common-protos [(#3955)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/3955) * Update dependency googleapis-common-protos to v1.52.0 * Update requirements.txt * Update requirements.txt * Update requirements.txt Co-authored-by: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com> * Replace GCLOUD_PROJECT with GOOGLE_CLOUD_PROJECT. [(#4022)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/4022) * [tasks] testing: use fixtures for the queue [(#4049)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/4049) fixes #4045 fixes #4044 I don't know why these tests started to fail, but anyways we'd better use fixtures and temporary queues. * add python snippets and tests for creating, listing, and deleting queues [(#4012)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/4012) * add python snippets and tests for creating, listing, and deleting queues * fix grammar * update licenses * apply suggested fixes and format with black * refine delete_queue_test with fixture for setup * utilize fixtures and match format of create_http_task_test * utilize fixtures in list_queues_test and create_queue_test * make create_queue_test call the right function * still attempt to delete queue after test runs in case of failure * attempt to delete queue in case of failure, using try/except approach * add print when NotFound is caught * fix import Co-authored-by: Averi Kitsch Co-authored-by: Takashi Matsuo * docs(tasks): service_account_email parameter example [(#4183)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/4183) * docs(tasks): serviceAccountEmail parameter example * docs(tasks): parameter name camel => snake case * docs(tasks): comment order = function param order * chore(deps): update dependency pytest to v5.4.3 [(#4279)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/4279) * chore(deps): update dependency pytest to v5.4.3 * specify pytest for python 2 in appengine Co-authored-by: Leah Cole * Update dependency pytest to v6 [(#4390)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/4390) * tasks: added json content-type request [(#4473)](https://github.com/GoogleCloudPlatform/python-docs-samples/issues/4473) - added json payload compatibility - fix imports and code block used on https://cloud.google.com/tasks/docs/creating-http-target-tasks#python ## Description Fixes # Note: It's a good idea to open an issue first for discussion. ## Checklist - [ ] I have followed [Sample Guidelines from AUTHORING_GUIDE.MD](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md) - [ ] README is updated to include [all relevant information](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md#readme-file) - [ ] **Tests** pass: `nox -s py-3.6` (see [Test Environment Setup](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md#test-environment-setup)) - [ ] **Lint** pass: `nox -s lint` (see [Test Environment Setup](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md#test-environment-setup)) - [ ] These samples need a new **API enabled** in testing projects to pass (let us know which ones) - [ ] These samples need a new/updated **env vars** in testing projects set to pass (let us know which ones) - [ ] Please **merge** this PR for me once it is approved. - [ ] This sample adds a new sample directory, and I updated the [CODEOWNERS file](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/.github/CODEOWNERS) with the codeowners for this sample * docs: add samples from python-docs-samples/tasks * chore: update templates Co-authored-by: Andrew Gorcester Co-authored-by: Andrew Gorcester Co-authored-by: DPE bot Co-authored-by: michaelawyu Co-authored-by: ellenevans <35748459+ellenevans@users.noreply.github.com> Co-authored-by: Averi Kitsch Co-authored-by: michaelawyu Co-authored-by: Noah Negrey Co-authored-by: Alex Voorhees Co-authored-by: Gus Class Co-authored-by: Sarath Kaul Co-authored-by: Sarath Kaul Co-authored-by: WhiteSource Renovate Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com> Co-authored-by: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com> Co-authored-by: Takashi Matsuo Co-authored-by: Aaron Johnson Co-authored-by: Adam Ross Co-authored-by: Leah Cole Co-authored-by: Joab Leite S. Neto --- cloud-tasks/snippets/create_http_task.py | 138 +++++++++++ cloud-tasks/snippets/create_http_task_test.py | 48 ++++ .../snippets/create_http_task_with_token.py | 86 +++++++ .../create_http_task_with_token_test.py | 53 +++++ cloud-tasks/snippets/create_queue.py | 35 +++ cloud-tasks/snippets/create_queue_test.py | 40 ++++ cloud-tasks/snippets/delete_queue.py | 30 +++ cloud-tasks/snippets/delete_queue_test.py | 56 +++++ cloud-tasks/snippets/list_queues.py | 36 +++ cloud-tasks/snippets/list_queues_test.py | 55 +++++ cloud-tasks/snippets/noxfile.py | 224 ++++++++++++++++++ cloud-tasks/snippets/requirements-test.txt | 1 + cloud-tasks/snippets/requirements.txt | 1 + 13 files changed, 803 insertions(+) create mode 100644 cloud-tasks/snippets/create_http_task.py create mode 100644 cloud-tasks/snippets/create_http_task_test.py create mode 100644 cloud-tasks/snippets/create_http_task_with_token.py create mode 100644 cloud-tasks/snippets/create_http_task_with_token_test.py create mode 100644 cloud-tasks/snippets/create_queue.py create mode 100644 cloud-tasks/snippets/create_queue_test.py create mode 100644 cloud-tasks/snippets/delete_queue.py create mode 100644 cloud-tasks/snippets/delete_queue_test.py create mode 100644 cloud-tasks/snippets/list_queues.py create mode 100644 cloud-tasks/snippets/list_queues_test.py create mode 100644 cloud-tasks/snippets/noxfile.py create mode 100644 cloud-tasks/snippets/requirements-test.txt create mode 100644 cloud-tasks/snippets/requirements.txt diff --git a/cloud-tasks/snippets/create_http_task.py b/cloud-tasks/snippets/create_http_task.py new file mode 100644 index 000000000000..23896209a104 --- /dev/null +++ b/cloud-tasks/snippets/create_http_task.py @@ -0,0 +1,138 @@ +# Copyright 2019 Google LLC All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import print_function + +import argparse + + +def create_http_task(project, + queue, + location, + url, + payload=None, + in_seconds=None, + task_name=None): + # [START cloud_tasks_create_http_task] + """Create a task for a given queue with an arbitrary payload.""" + + from google.cloud import tasks_v2 + from google.protobuf import timestamp_pb2 + import datetime + import json + + # Create a client. + client = tasks_v2.CloudTasksClient() + + # TODO(developer): Uncomment these lines and replace with your values. + # project = 'my-project-id' + # queue = 'my-queue' + # location = 'us-central1' + # url = 'https://example.com/task_handler' + # payload = 'hello' or {'param': 'value'} for application/json + + # Construct the fully qualified queue name. + parent = client.queue_path(project, location, queue) + + # Construct the request body. + task = { + 'http_request': { # Specify the type of request. + 'http_method': 'POST', + 'url': url # The full url path that the task will be sent to. + } + } + if payload is not None: + if isinstance(payload, dict): + # Convert dict to JSON string + payload = json.dumps(payload) + # specify http content-type to application/json + task['http_request']['headers'] = {'Content-type': 'application/json'} + + # The API expects a payload of type bytes. + converted_payload = payload.encode() + + # Add the payload to the request. + task['http_request']['body'] = converted_payload + + if in_seconds is not None: + # Convert "seconds from now" into an rfc3339 datetime string. + d = datetime.datetime.utcnow() + datetime.timedelta(seconds=in_seconds) + + # Create Timestamp protobuf. + timestamp = timestamp_pb2.Timestamp() + timestamp.FromDatetime(d) + + # Add the timestamp to the tasks. + task['schedule_time'] = timestamp + + if task_name is not None: + # Add the name to tasks. + task['name'] = task_name + + # Use the client to build and send the task. + response = client.create_task(parent, task) + + print('Created task {}'.format(response.name)) + # [END cloud_tasks_create_http_task] + return response + + +if __name__ == '__main__': + parser = argparse.ArgumentParser( + description=create_http_task.__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + + parser.add_argument( + '--project', + help='Project of the queue to add the task to.', + required=True, + ) + + parser.add_argument( + '--queue', + help='ID (short name) of the queue to add the task to.', + required=True, + ) + + parser.add_argument( + '--location', + help='Location of the queue to add the task to.', + required=True, + ) + + parser.add_argument( + '--url', + help='The full url path that the request will be sent to.', + required=True, + ) + + parser.add_argument( + '--payload', + help='Optional payload to attach to the push queue.' + ) + + parser.add_argument( + '--in_seconds', type=int, + help='The number of seconds from now to schedule task attempt.' + ) + + parser.add_argument( + '--task_name', + help='Task name of the task to create' + ) + args = parser.parse_args() + + create_http_task( + args.project, args.queue, args.location, args.url, + args.payload, args.in_seconds, args.task_name) diff --git a/cloud-tasks/snippets/create_http_task_test.py b/cloud-tasks/snippets/create_http_task_test.py new file mode 100644 index 000000000000..b0fb3ed758b0 --- /dev/null +++ b/cloud-tasks/snippets/create_http_task_test.py @@ -0,0 +1,48 @@ +# Copyright 2019 Google LLC All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +import uuid + +from google.cloud import tasks_v2 +import pytest + +import create_http_task + +TEST_PROJECT_ID = os.getenv('GOOGLE_CLOUD_PROJECT') +TEST_LOCATION = os.getenv('TEST_QUEUE_LOCATION', 'us-central1') +TEST_QUEUE_NAME = f'my-queue-{uuid.uuid4().hex}' + + +@pytest.fixture() +def test_queue(): + client = tasks_v2.CloudTasksClient() + parent = client.location_path(TEST_PROJECT_ID, TEST_LOCATION) + queue = { + # The fully qualified path to the queue + 'name': client.queue_path( + TEST_PROJECT_ID, TEST_LOCATION, TEST_QUEUE_NAME), + } + q = client.create_queue(parent, queue) + + yield q + + client.delete_queue(q.name) + + +def test_create_http_task(test_queue): + url = 'https://example.com/task_handler' + result = create_http_task.create_http_task( + TEST_PROJECT_ID, TEST_QUEUE_NAME, TEST_LOCATION, url) + assert TEST_QUEUE_NAME in result.name diff --git a/cloud-tasks/snippets/create_http_task_with_token.py b/cloud-tasks/snippets/create_http_task_with_token.py new file mode 100644 index 000000000000..7320ede3a998 --- /dev/null +++ b/cloud-tasks/snippets/create_http_task_with_token.py @@ -0,0 +1,86 @@ +# Copyright 2019 Google LLC All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import print_function + +import datetime + + +def create_http_task(project, + queue, + location, + url, + service_account_email, + payload=None, + in_seconds=None, + task_name=None): + # [START cloud_tasks_create_http_task_with_token] + """Create a task for a given queue with an arbitrary payload.""" + + from google.cloud import tasks_v2 + from google.protobuf import timestamp_pb2 + + # Create a client. + client = tasks_v2.CloudTasksClient() + + # TODO(developer): Uncomment these lines and replace with your values. + # project = 'my-project-id' + # queue = 'my-queue' + # location = 'us-central1' + # url = 'https://example.com/task_handler' + # service_account_email = 'service-account@my-project-id.iam.gserviceaccount.com'; + # payload = 'hello' + + # Construct the fully qualified queue name. + parent = client.queue_path(project, location, queue) + + # Construct the request body. + task = { + 'http_request': { # Specify the type of request. + 'http_method': 'POST', + 'url': url, # The full url path that the task will be sent to. + 'oidc_token': { + 'service_account_email': service_account_email + } + } + } + + if payload is not None: + # The API expects a payload of type bytes. + converted_payload = payload.encode() + + # Add the payload to the request. + task['http_request']['body'] = converted_payload + + if in_seconds is not None: + # Convert "seconds from now" into an rfc3339 datetime string. + d = datetime.datetime.utcnow() + datetime.timedelta(seconds=in_seconds) + + # Create Timestamp protobuf. + timestamp = timestamp_pb2.Timestamp() + timestamp.FromDatetime(d) + + # Add the timestamp to the tasks. + task['schedule_time'] = timestamp + + if task_name is not None: + # Add the name to tasks. + task['name'] = task_name + + # Use the client to build and send the task. + response = client.create_task(parent, task) + + print('Created task {}'.format(response.name)) + return response +# [END cloud_tasks_create_http_task_with_token] diff --git a/cloud-tasks/snippets/create_http_task_with_token_test.py b/cloud-tasks/snippets/create_http_task_with_token_test.py new file mode 100644 index 000000000000..dd90d9199b1e --- /dev/null +++ b/cloud-tasks/snippets/create_http_task_with_token_test.py @@ -0,0 +1,53 @@ +# Copyright 2019 Google LLC All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +import uuid + +from google.cloud import tasks_v2 +import pytest + +import create_http_task_with_token + +TEST_PROJECT_ID = os.getenv('GOOGLE_CLOUD_PROJECT') +TEST_LOCATION = os.getenv('TEST_QUEUE_LOCATION', 'us-central1') +TEST_QUEUE_NAME = f'my-queue-{uuid.uuid4().hex}' +TEST_SERVICE_ACCOUNT = ( + 'test-run-invoker@python-docs-samples-tests.iam.gserviceaccount.com') + + +@pytest.fixture() +def test_queue(): + client = tasks_v2.CloudTasksClient() + parent = client.location_path(TEST_PROJECT_ID, TEST_LOCATION) + queue = { + # The fully qualified path to the queue + 'name': client.queue_path( + TEST_PROJECT_ID, TEST_LOCATION, TEST_QUEUE_NAME), + } + q = client.create_queue(parent, queue) + + yield q + + client.delete_queue(q.name) + + +def test_create_http_task_with_token(test_queue): + url = 'https://example.com/task_handler' + result = create_http_task_with_token.create_http_task(TEST_PROJECT_ID, + TEST_QUEUE_NAME, + TEST_LOCATION, + url, + TEST_SERVICE_ACCOUNT) + assert TEST_QUEUE_NAME in result.name diff --git a/cloud-tasks/snippets/create_queue.py b/cloud-tasks/snippets/create_queue.py new file mode 100644 index 000000000000..d8d4dfaea263 --- /dev/null +++ b/cloud-tasks/snippets/create_queue.py @@ -0,0 +1,35 @@ +# Copyright 2020 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# [START cloud_tasks_create_queue] +def create_queue(project, queue_name, location): + """Create a task queue.""" + + from google.cloud import tasks_v2 + + # Create a client. + client = tasks_v2.CloudTasksClient() + + # Construct the fully qualified location path. + parent = client.location_path(project, location) + + # Construct the create queue request. + queue = {'name': client.queue_path(project, location, queue_name)} + + # Use the client to create the queue. + response = client.create_queue(parent, queue) + + print('Created queue {}'.format(response.name)) + return response +# [END cloud_tasks_create_queue] diff --git a/cloud-tasks/snippets/create_queue_test.py b/cloud-tasks/snippets/create_queue_test.py new file mode 100644 index 000000000000..1f623a2aa6cd --- /dev/null +++ b/cloud-tasks/snippets/create_queue_test.py @@ -0,0 +1,40 @@ +# Copyright 2020 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +import uuid + +from google.cloud import tasks_v2 +import pytest + +import create_queue + +TEST_PROJECT_ID = os.environ['GOOGLE_CLOUD_PROJECT'] +TEST_LOCATION = os.getenv('TEST_QUEUE_LOCATION', 'us-central1') +TEST_QUEUE_NAME = f'my-queue-{uuid.uuid4().hex}' + + +@pytest.fixture() +def test_queue(): + client = tasks_v2.CloudTasksClient() + q = create_queue.create_queue(TEST_PROJECT_ID, TEST_QUEUE_NAME, TEST_LOCATION) + + yield q + + client.delete_queue(q.name) + + +def test_create_queue(capsys, test_queue): + out, _ = capsys.readouterr() + assert 'Created queue' in out diff --git a/cloud-tasks/snippets/delete_queue.py b/cloud-tasks/snippets/delete_queue.py new file mode 100644 index 000000000000..d69514249a17 --- /dev/null +++ b/cloud-tasks/snippets/delete_queue.py @@ -0,0 +1,30 @@ +# Copyright 2020 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# [START cloud_tasks_delete_queue] +def delete_queue(project, queue_name, location): + """Delete a task queue.""" + + from google.cloud import tasks_v2 + + # Create a client. + client = tasks_v2.CloudTasksClient() + + # Get the fully qualified path to queue. + queue = client.queue_path(project, location, queue_name) + + # Use the client to delete the queue. + client.delete_queue(queue) + print('Deleted queue') +# [END cloud_tasks_delete_queue] diff --git a/cloud-tasks/snippets/delete_queue_test.py b/cloud-tasks/snippets/delete_queue_test.py new file mode 100644 index 000000000000..4e1acf403dac --- /dev/null +++ b/cloud-tasks/snippets/delete_queue_test.py @@ -0,0 +1,56 @@ +# Copyright 2020 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +import uuid + +from google.api_core import exceptions +from google.cloud import tasks_v2 +import pytest + +import delete_queue + + +TEST_PROJECT_ID = os.environ['GOOGLE_CLOUD_PROJECT'] +TEST_LOCATION = os.getenv('TEST_QUEUE_LOCATION', 'us-central1') +TEST_QUEUE_NAME = f'my-queue-{uuid.uuid4().hex}' + + +@pytest.fixture() +def test_queue(): + client = tasks_v2.CloudTasksClient() + parent = client.location_path(TEST_PROJECT_ID, TEST_LOCATION) + queue = { + # The fully qualified path to the queue + 'name': client.queue_path( + TEST_PROJECT_ID, TEST_LOCATION, TEST_QUEUE_NAME), + } + q = client.create_queue(parent, queue) + + yield q + + try: + # Attempt to delete the queue in case the sample failed. + client.delete_queue(q.name) + except exceptions.NotFound: + # The queue was already successfully deleted. + print('Queue already deleted successfully') + + +def test_delete_queue(capsys, test_queue): + delete_queue.delete_queue( + TEST_PROJECT_ID, TEST_QUEUE_NAME, TEST_LOCATION + ) + out, _ = capsys.readouterr() + assert 'Deleted queue' in out diff --git a/cloud-tasks/snippets/list_queues.py b/cloud-tasks/snippets/list_queues.py new file mode 100644 index 000000000000..fa48907fad17 --- /dev/null +++ b/cloud-tasks/snippets/list_queues.py @@ -0,0 +1,36 @@ +# Copyright 2020 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# [START cloud_tasks_list_queues] +def list_queues(project, location): + """List all task queues.""" + + from google.cloud import tasks_v2 + + # Create a client. + client = tasks_v2.CloudTasksClient() + + # Construct the fully qualified location path. + parent = client.location_path(project, location) + + # Use the client to obtain the queues. + response = client.list_queues(parent) + + # Print the results. + for queue in response: + print(queue.name) + + if response.num_results == 0: + print('No queues found!') +# [END cloud_tasks_list_queues] diff --git a/cloud-tasks/snippets/list_queues_test.py b/cloud-tasks/snippets/list_queues_test.py new file mode 100644 index 000000000000..df11d0bd4f9b --- /dev/null +++ b/cloud-tasks/snippets/list_queues_test.py @@ -0,0 +1,55 @@ +# Copyright 2020 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +import uuid + +from google.cloud import tasks_v2 +import pytest + +import list_queues + +TEST_PROJECT_ID = os.environ['GOOGLE_CLOUD_PROJECT'] +TEST_LOCATION = os.getenv('TEST_QUEUE_LOCATION', 'us-central1') +TEST_QUEUE_NAME = f'my-queue-{uuid.uuid4().hex}' + + +@pytest.fixture() +def test_queue(): + client = tasks_v2.CloudTasksClient() + parent = client.location_path(TEST_PROJECT_ID, TEST_LOCATION) + queue = { + # The fully qualified path to the queue + 'name': client.queue_path( + TEST_PROJECT_ID, TEST_LOCATION, TEST_QUEUE_NAME), + } + q = client.create_queue(parent, queue) + + yield q + + client.delete_queue(q.name) + + +def test_list_queues_not_present(capsys): + list_queues.list_queues(TEST_PROJECT_ID, TEST_LOCATION) + out, _ = capsys.readouterr() + + assert(TEST_QUEUE_NAME not in out) + + +def test_list_queues_present(capsys, test_queue): + list_queues.list_queues(TEST_PROJECT_ID, TEST_LOCATION) + out, _ = capsys.readouterr() + + assert(TEST_QUEUE_NAME in out) diff --git a/cloud-tasks/snippets/noxfile.py b/cloud-tasks/snippets/noxfile.py new file mode 100644 index 000000000000..ba55d7ce53ca --- /dev/null +++ b/cloud-tasks/snippets/noxfile.py @@ -0,0 +1,224 @@ +# Copyright 2019 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import print_function + +import os +from pathlib import Path +import sys + +import nox + + +# WARNING - WARNING - WARNING - WARNING - WARNING +# WARNING - WARNING - WARNING - WARNING - WARNING +# DO NOT EDIT THIS FILE EVER! +# WARNING - WARNING - WARNING - WARNING - WARNING +# WARNING - WARNING - WARNING - WARNING - WARNING + +# Copy `noxfile_config.py` to your directory and modify it instead. + + +# `TEST_CONFIG` dict is a configuration hook that allows users to +# modify the test configurations. The values here should be in sync +# with `noxfile_config.py`. Users will copy `noxfile_config.py` into +# their directory and modify it. + +TEST_CONFIG = { + # You can opt out from the test for specific Python versions. + 'ignored_versions': ["2.7"], + + # An envvar key for determining the project id to use. Change it + # to 'BUILD_SPECIFIC_GCLOUD_PROJECT' if you want to opt in using a + # build specific Cloud project. You can also use your own string + # to use your own Cloud project. + 'gcloud_project_env': 'GOOGLE_CLOUD_PROJECT', + # 'gcloud_project_env': 'BUILD_SPECIFIC_GCLOUD_PROJECT', + + # A dictionary you want to inject into your test. Don't put any + # secrets here. These values will override predefined values. + 'envs': {}, +} + + +try: + # Ensure we can import noxfile_config in the project's directory. + sys.path.append('.') + from noxfile_config import TEST_CONFIG_OVERRIDE +except ImportError as e: + print("No user noxfile_config found: detail: {}".format(e)) + TEST_CONFIG_OVERRIDE = {} + +# Update the TEST_CONFIG with the user supplied values. +TEST_CONFIG.update(TEST_CONFIG_OVERRIDE) + + +def get_pytest_env_vars(): + """Returns a dict for pytest invocation.""" + ret = {} + + # Override the GCLOUD_PROJECT and the alias. + env_key = TEST_CONFIG['gcloud_project_env'] + # This should error out if not set. + ret['GOOGLE_CLOUD_PROJECT'] = os.environ[env_key] + + # Apply user supplied envs. + ret.update(TEST_CONFIG['envs']) + return ret + + +# DO NOT EDIT - automatically generated. +# All versions used to tested samples. +ALL_VERSIONS = ["2.7", "3.6", "3.7", "3.8"] + +# Any default versions that should be ignored. +IGNORED_VERSIONS = TEST_CONFIG['ignored_versions'] + +TESTED_VERSIONS = sorted([v for v in ALL_VERSIONS if v not in IGNORED_VERSIONS]) + +INSTALL_LIBRARY_FROM_SOURCE = bool(os.environ.get("INSTALL_LIBRARY_FROM_SOURCE", False)) +# +# Style Checks +# + + +def _determine_local_import_names(start_dir): + """Determines all import names that should be considered "local". + + This is used when running the linter to insure that import order is + properly checked. + """ + file_ext_pairs = [os.path.splitext(path) for path in os.listdir(start_dir)] + return [ + basename + for basename, extension in file_ext_pairs + if extension == ".py" + or os.path.isdir(os.path.join(start_dir, basename)) + and basename not in ("__pycache__") + ] + + +# Linting with flake8. +# +# We ignore the following rules: +# E203: whitespace before ‘:’ +# E266: too many leading ‘#’ for block comment +# E501: line too long +# I202: Additional newline in a section of imports +# +# We also need to specify the rules which are ignored by default: +# ['E226', 'W504', 'E126', 'E123', 'W503', 'E24', 'E704', 'E121'] +FLAKE8_COMMON_ARGS = [ + "--show-source", + "--builtin=gettext", + "--max-complexity=20", + "--import-order-style=google", + "--exclude=.nox,.cache,env,lib,generated_pb2,*_pb2.py,*_pb2_grpc.py", + "--ignore=E121,E123,E126,E203,E226,E24,E266,E501,E704,W503,W504,I202", + "--max-line-length=88", +] + + +@nox.session +def lint(session): + session.install("flake8", "flake8-import-order") + + local_names = _determine_local_import_names(".") + args = FLAKE8_COMMON_ARGS + [ + "--application-import-names", + ",".join(local_names), + "." + ] + session.run("flake8", *args) + + +# +# Sample Tests +# + + +PYTEST_COMMON_ARGS = ["--junitxml=sponge_log.xml"] + + +def _session_tests(session, post_install=None): + """Runs py.test for a particular project.""" + if os.path.exists("requirements.txt"): + session.install("-r", "requirements.txt") + + if os.path.exists("requirements-test.txt"): + session.install("-r", "requirements-test.txt") + + if INSTALL_LIBRARY_FROM_SOURCE: + session.install("-e", _get_repo_root()) + + if post_install: + post_install(session) + + session.run( + "pytest", + *(PYTEST_COMMON_ARGS + session.posargs), + # Pytest will return 5 when no tests are collected. This can happen + # on travis where slow and flaky tests are excluded. + # See http://doc.pytest.org/en/latest/_modules/_pytest/main.html + success_codes=[0, 5], + env=get_pytest_env_vars() + ) + + +@nox.session(python=ALL_VERSIONS) +def py(session): + """Runs py.test for a sample using the specified version of Python.""" + if session.python in TESTED_VERSIONS: + _session_tests(session) + else: + session.skip("SKIPPED: {} tests are disabled for this sample.".format( + session.python + )) + + +# +# Readmegen +# + + +def _get_repo_root(): + """ Returns the root folder of the project. """ + # Get root of this repository. Assume we don't have directories nested deeper than 10 items. + p = Path(os.getcwd()) + for i in range(10): + if p is None: + break + if Path(p / ".git").exists(): + return str(p) + p = p.parent + raise Exception("Unable to detect repository root.") + + +GENERATED_READMES = sorted([x for x in Path(".").rglob("*.rst.in")]) + + +@nox.session +@nox.parametrize("path", GENERATED_READMES) +def readmegen(session, path): + """(Re-)generates the readme for a sample.""" + session.install("jinja2", "pyyaml") + dir_ = os.path.dirname(path) + + if os.path.exists(os.path.join(dir_, "requirements.txt")): + session.install("-r", os.path.join(dir_, "requirements.txt")) + + in_file = os.path.join(dir_, "README.rst.in") + session.run( + "python", _get_repo_root() + "/scripts/readme-gen/readme_gen.py", in_file + ) diff --git a/cloud-tasks/snippets/requirements-test.txt b/cloud-tasks/snippets/requirements-test.txt new file mode 100644 index 000000000000..7e460c8c866e --- /dev/null +++ b/cloud-tasks/snippets/requirements-test.txt @@ -0,0 +1 @@ +pytest==6.0.1 diff --git a/cloud-tasks/snippets/requirements.txt b/cloud-tasks/snippets/requirements.txt new file mode 100644 index 000000000000..b7bc58a3b74e --- /dev/null +++ b/cloud-tasks/snippets/requirements.txt @@ -0,0 +1 @@ +google-cloud-tasks==1.5.0