Implementation of the Checkpointstoragetable

sdk/eventhub/azure-eventhub-checkpointstoretable/CHANGELOG.md

@@ -0,0 +1,4 @@
+# Release History
+
+## 1.0.0b1 (Unreleased)
+  - Template package (can update this after implementation is done)

sdk/eventhub/azure-eventhub-checkpointstoretable/MANIFEST.in

@@ -0,0 +1,6 @@
+include *.md
+include azure/__init__.py
+include azure/eventhub/__init__.py
+include azure/eventhub/extensions/__init__.py
+recursive-include tests *.py
+recursive-include samples *.py

sdk/eventhub/azure-eventhub-checkpointstoretable/README.md

@@ -0,0 +1,41 @@
+[![Build Status](https://dev.azure.com/azure-sdk/public/_apis/build/status/azure-sdk-for-python.client?branchName=main)](https://dev.azure.com/azure-sdk/public/_build/latest?definitionId=46?branchName=main)
+
+# Azure EventHubs Checkpoint Store client library for Python using Tables
+
+Azure EventHubs Checkpoint Store is used for storing checkpoints while processing events from Azure Event Hubs.
+This Checkpoint Store package works as a plug-in package to `EventHubConsumerClient`. It uses Azure Tables as the persistent store for maintaining checkpoints and partition ownership information.
+
+
+# Getting started
+
+### Prerequisites
+
+- Python2.7, Python 3.6 or later.
+- **Microsoft Azure Subscription:**  To use Azure services, including Azure Event Hubs, you'll need a subscription. If you do not have an existing Azure account, you may sign up for a free trial or use your MSDN subscriber benefits when you [create an account](https://azure.microsoft.com/).
+
+- **Event Hubs namespace with an Event Hub:** To interact with Azure Event Hubs, you'll also need to have a namespace and Event Hub  available.  If you are not familiar with creating Azure resources, you may wish to follow the step-by-step guide for [creating an Event Hub using the Azure portal](https://docs.microsoft.com/azure/event-hubs/event-hubs-create).  There, you can also find detailed instructions for using the Azure CLI, Azure PowerShell, or Azure Resource Manager (ARM) templates to create an Event Hub.
+
+- **Azure Storage Account:** You'll need to have an Azure Storage Account and create a Azure Table Storage to store the checkpoint data with entities. You may follow the guide [creating an Azure Table Storage Account]
+(https://docs.microsoft.com/azure/storage/tables/table-storage-overview).
+
+# Key concepts
+
+Bullet point list of your library's main concepts.
+
+# Examples
+
+Examples of some of the key concepts for your library.
+
+# Troubleshooting
+
+Running into issues? This section should contain details as to what to do there.
+
+# Next steps
+
+More sample code should go here, along with links out to the appropriate example tests.
+
+# Contributing
+
+If you encounter any bugs or have suggestions, please file an issue in the [Issues](<https://github.com/Azure/azure-sdk-for-python/issues>) section of the project.
+
+![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-python%2Fsdk%2Feventhub%2Fazure-eventhub-checkpointstoretable%2FREADME.png)

sdk/eventhub/azure-eventhub-checkpointstoretable/azure/__init__.py

@@ -0,0 +1 @@
+__path__ = __import__("pkgutil").extend_path(__path__, __name__)

sdk/eventhub/azure-eventhub-checkpointstoretable/azure/eventhub/__init__.py

@@ -0,0 +1,5 @@
+# --------------------------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for license information.
+# --------------------------------------------------------------------------------------------
+__path__ = __import__("pkgutil").extend_path(__path__, __name__)  # type: ignore

sdk/eventhub/azure-eventhub-checkpointstoretable/azure/eventhub/extensions/__init__.py

@@ -0,0 +1,5 @@
+# --------------------------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for license information.
+# --------------------------------------------------------------------------------------------
+__path__ = __import__("pkgutil").extend_path(__path__, __name__)  # type: ignore

sdk/eventhub/azure-eventhub-checkpointstoretable/azure/eventhub/extensions/checkpointstoretable/__init__.py

@@ -0,0 +1,14 @@
+# --------------------------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for license information.
+# --------------------------------------------------------------------------------------------
+
+from ._version import VERSION
+
+__version__ = VERSION
+
+from ._tablestoragecs import TableCheckpointStore
+
+__all__ = [
+    "TableCheckpointStore",
+]

sdk/eventhub/azure-eventhub-checkpointstoretable/azure/eventhub/extensions/checkpointstoretable/_version.py

@@ -0,0 +1,6 @@
+# ------------------------------------
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+# ------------------------------------
+
+VERSION = "1.0.0b1"

sdk/eventhub/azure-eventhub-checkpointstoretable/dev_requirements.txt

@@ -0,0 +1,3 @@
+-e ../../../tools/azure-sdk-tools
+../../core/azure-core
+-e ../../../tools/azure-devtools

sdk/eventhub/azure-eventhub-checkpointstoretable/sdk_packaging.toml

@@ -0,0 +1,2 @@
+[packaging]
+auto_update = false

sdk/eventhub/azure-eventhub-checkpointstoretable/setup.cfg

@@ -0,0 +1,2 @@
+[bdist_wheel]
+universal=1

sdk/eventhub/azure-eventhub-checkpointstoretable/setup.py

@@ -0,0 +1,76 @@
+from setuptools import setup, find_packages
+import os
+from io import open
+import re
+
+# example setup.py Feel free to copy the entire "azure-template" folder into a package folder named
+# with "azure-<yourpackagename>". Ensure that the below arguments to setup() are updated to reflect
+# your package.
+
+# this setup.py is set up in a specific way to keep the azure* and azure-mgmt-* namespaces WORKING all the way
+# up from python 2.7. Reference here: https://github.com/Azure/azure-sdk-for-python/wiki/Azure-packaging
+
+PACKAGE_NAME = "azure-eventhub-checkpointstoretable"
+PACKAGE_PPRINT_NAME = "Event Hubs checkpointer implementation with Azure Table Storage"
+
+# a-b-c => a/b/c
+package_folder_path = PACKAGE_NAME.replace("-", "/")
+# a-b-c => a.b.c
+namespace_name = PACKAGE_NAME.replace("-", ".")
+
+package_folder_path = "azure/eventhub/extensions/checkpointstoretable"
+
+# Version extraction inspired from 'requests'
+with open(os.path.join(package_folder_path, "_version.py"), "r") as fd:
+    version = re.search(r'^VERSION\s*=\s*[\'"]([^\'"]*)[\'"]', fd.read(), re.MULTILINE).group(1)
+if not version:
+    raise RuntimeError("Cannot find version information")
+
+with open("README.md", encoding="utf-8") as f:
+    long_description = f.read()
+
+setup(
+    name=PACKAGE_NAME,
+    version=version,
+    description="Microsoft Azure {} Client Library for Python".format(PACKAGE_PPRINT_NAME),
+    # ensure that these are updated to reflect the package owners' information
+    long_description=long_description,
+    long_description_content_type="text/markdown",
+    url="https://github.com/Azure/azure-sdk-for-python",
+    author="Microsoft Corporation",
+    author_email="azuresdkengsysadmins@microsoft.com",
+    license="MIT License",
+    # ensure that the development status reflects the status of your package
+    classifiers=[
+        "Development Status :: 4 - Beta",
+        "Programming Language :: Python",
+        "Programming Language :: Python :: 2",
+        "Programming Language :: Python :: 2.7",
+        "Programming Language :: Python :: 3",
+        "Programming Language :: Python :: 3.6",
+        "Programming Language :: Python :: 3.7",
+        "Programming Language :: Python :: 3.8",
+        "Programming Language :: Python :: 3.9",
+        "License :: OSI Approved :: MIT License",
+    ],
+    packages=find_packages(
+        exclude=[
+            "tests",
+            # Exclude packages that will be covered by PEP420 or nspkg
+            # This means any folder structure that only consists of a __init__.py.
+            # For example, for storage, this would mean adding 'azure.storage'
+            # in addition to the default 'azure' that is seen here.
+            "azure",
+        ]
+    ),
+    install_requires=[
+        "azure-core<2.0.0,>=1.2.2",
+    ],
+    extras_require={
+        ":python_version<'3.0'": ["azure-nspkg"],
+    },
+    project_urls={
+        "Bug Reports": "https://github.com/Azure/azure-sdk-for-python/issues",
+        "Source": "https://github.com/Azure/azure-sdk-python",
+    },
+)

sdk/eventhub/azure-eventhub-checkpointstoretable/tests/test_storage_table_partition_manager.py

@@ -0,0 +1,7 @@
+import pytest
+from azure.eventhub.extensions.checkpointstoretable import TableCheckpointStore
+
+def test_constructor():
+    client = TableCheckpointStore()
+    assert client is not None
+

sdk/eventhub/ci.yml

@@ -44,5 +44,7 @@ extends:
       safeName: azureeventhubcheckpointstoreblobaio
     - name: azure-eventhub-checkpointstoreblob
       safeName: azureeventhubcheckpointstoreblob
+    - name: azure-eventhub-checkpointstoretable
+      safeName: azureeventhubcheckpointstoretable
     - name: azure-mgmt-eventhub
       safeName: azuremgmteventhub