Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
…into regenerate_keys

* 'master' of https://github.com/Azure/azure-sdk-for-python:
  don't use mgmt track2 (Azure#12183)
  Fix pip version requirement in build-test.yml (Azure#12148)
  Revert "Resolve Pip Related Errors (Azure#12157)", Pin VirtualEnv (Azure#12169)
  Smart text serialization (Azure#12149)
  Remove OSName Variable (Azure#12147)
  [formrecognizer] add strongly-typed receipt wrapper sample (Azure#12128)
  Add missing __init__ type annotations (Azure#12146)
  fix sample in readme (Azure#12144)
  • Loading branch information
iscai-msft committed Jun 24, 2020
2 parents 3ac2aea + 7b8084b commit c3a2d16
Show file tree
Hide file tree
Showing 24 changed files with 351 additions and 27 deletions.
5 changes: 3 additions & 2 deletions eng/ci_tools.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# requirements leveraged by ci tools
setuptools==44.1.0; python_version == '2.7'
setuptools==45.1.0; python_version >= '3.5'
wheel==0.34.2
virtualenv==20.0.23
wheel==0.34.2
Jinja2==2.11.1
packaging==20.4
tox==3.14.6
Expand All @@ -11,7 +12,7 @@ pathlib2==2.3.5
readme-renderer[md]==25.0
doc-warden==0.5.4
coverage==4.5.4
codecov==2.0.22
codecov==2.0.22
beautifulsoup4==4.8.2
pkginfo==1.5.0.1

Expand Down
2 changes: 1 addition & 1 deletion eng/pipelines/templates/steps/build-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ steps:
- template: /eng/common/pipelines/templates/steps/verify-agent-os.yml

- script: |
python -m pip install pip == 20.1
python -m pip install pip==20.1
pip install -r eng/ci_tools.txt
pip --version
displayName: 'Prep Environment'
Expand Down
1 change: 1 addition & 0 deletions sdk/core/azure-core/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

- `AzureKeyCredentialPolicy` will now accept (and ignore) passed in kwargs #11963
- Better error messages if passed endpoint is incorrect #12106
- Do not JSON encore a string if content type is "text" #12137

## 1.6.0 (2020-06-03)

Expand Down
25 changes: 25 additions & 0 deletions sdk/core/azure-core/azure/core/pipeline/transport/_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,19 @@ def set_streamed_data_body(self, data):
self.data = data
self.files = None

def set_text_body(self, data):
"""Set a text as body of the request.
:param data: A text to send as body.
:type data: str
"""
if data is None:
self.data = None
else:
self.data = data
self.headers["Content-Length"] = str(len(self.data))
self.files = None

def set_xml_body(self, data):
"""Set an XML element tree as the body of the request.
Expand Down Expand Up @@ -685,6 +698,11 @@ def _request(
# type: (...) -> HttpRequest
"""Create HttpRequest object.
If content is not None, guesses will be used to set the right body:
- If content is an XML tree, will serialize as XML
- If content-type starts by "text/", set the content as text
- Else, try JSON serialization
:param str method: HTTP method (GET, HEAD, etc.)
:param str url: URL for the request.
:param dict params: URL query parameters.
Expand All @@ -703,8 +721,15 @@ def _request(
request.headers.update(headers)

if content is not None:
content_type = request.headers.get("Content-Type")
if isinstance(content, ET.Element):
request.set_xml_body(content)
# https://github.com/Azure/azure-sdk-for-python/issues/12137
# A string is valid JSON, make the difference between text
# and a plain JSON string.
# Content-Type is a good indicator of intent from user
elif content_type and content_type.startswith("text/"):
request.set_text_body(content)
else:
try:
request.set_json_body(content)
Expand Down
19 changes: 19 additions & 0 deletions sdk/core/azure-core/tests/test_pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,25 @@ def test_request_url_with_params(self):

self.assertIn(request.url, ["a/b/c?g=h&t=y", "a/b/c?t=y&g=h"])

def test_request_text(self):
client = PipelineClientBase('http://example.org')
request = client.get(
"/",
content="foo"
)

# In absence of information, everything is JSON (double quote added)
assert request.data == json.dumps("foo")

request = client.post(
"/",
headers={'content-type': 'text/whatever'},
content="foo"
)

# We want a direct string
assert request.data == "foo"


if __name__ == "__main__":
unittest.main()
4 changes: 0 additions & 4 deletions sdk/eventhub/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,15 @@ jobs:
ServiceDirectory: eventhub
Matrix:
Linux_Python35:
OSName: 'Linux'
OSVmImage: 'ubuntu-18.04'
PythonVersion: '3.5'
Linux_Python38:
OSName: 'Linux'
OSVmImage: 'ubuntu-18.04'
PythonVersion: '3.8'
Windows_Python27:
OSName: 'Windows'
OSVmImage: 'windows-2019'
PythonVersion: '2.7'
MacOs_Python37:
OSName: 'MacOS'
OSVmImage: 'macOS-10.15'
PythonVersion: '3.7'
EnvVars:
Expand Down
2 changes: 1 addition & 1 deletion sdk/formrecognizer/azure-ai-formrecognizer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,7 @@ credential = AzureKeyCredential("<api_key>")
form_training_client = FormTrainingClient(endpoint, credential)

container_sas_url = "xxx" # training documents uploaded to blob storage
poller = form_training_client.begin_training(container_sas_url)
poller = form_training_client.begin_training(container_sas_url, use_training_labels=False)
model = poller.result()

# Custom model information
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,9 @@ def begin_recognize_receipts(self, receipt, **kwargs):
The input document must be of one of the supported content types - 'application/pdf',
'image/jpeg', 'image/png' or 'image/tiff'.
See fields found on a receipt here:
https://aka.ms/azsdk/python/formrecognizer/receiptfields
:param receipt: JPEG, PNG, PDF and TIFF type file stream or bytes.
Currently only supports US sales receipts.
:type receipt: bytes or IO[bytes]
Expand Down Expand Up @@ -141,6 +144,9 @@ def begin_recognize_receipts_from_url(self, receipt_url, **kwargs):
"""Extract field text and semantic values from a given US sales receipt.
The input document must be the location (Url) of the receipt to be analyzed.
See fields found on a receipt here:
https://aka.ms/azsdk/python/formrecognizer/receiptfields
:param str receipt_url: The url of the receipt to analyze. The input must be a valid, encoded url
of one of the supported formats: JPEG, PNG, PDF and TIFF. Currently only supports
US sales receipts.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,9 @@ async def begin_recognize_receipts(
The input document must be of one of the supported content types - 'application/pdf',
'image/jpeg', 'image/png' or 'image/tiff'.
See fields found on a receipt here:
https://aka.ms/azsdk/python/formrecognizer/receiptfields
:param receipt: JPEG, PNG, PDF and TIFF type file stream or bytes.
Currently only supports US sales receipts.
:type receipt: bytes or IO[bytes]
Expand Down Expand Up @@ -155,6 +158,9 @@ async def begin_recognize_receipts_from_url(
"""Extract field text and semantic values from a given US sales receipt.
The input document must be the location (Url) of the receipt to be analyzed.
See fields found on a receipt here:
https://aka.ms/azsdk/python/formrecognizer/receiptfields
:param str receipt_url: The url of the receipt to analyze. The input must be a valid, encoded url
of one of the supported formats: JPEG, PNG, PDF and TIFF. Currently only supports
US sales receipts.
Expand Down
5 changes: 4 additions & 1 deletion sdk/formrecognizer/azure-ai-formrecognizer/samples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ what you can do with the Azure Form Recognizer client library.

|**Advanced Sample File Name**|**Description**|
|----------------|-------------|
|[sample_strongly_typing_recognized_form.py][sample_strongly_typing_recognized_form] and [sample_strongly_typing_recognized_form_async.py][sample_strongly_typing_recognized_form_async]|Use the fields in your recognized forms to create an object with strongly-typed fields|
|[sample_get_bounding_boxes.py][sample_get_bounding_boxes] and [sample_get_bounding_boxes_async.py][sample_get_bounding_boxes_async]|Get info to visualize the outlines of form content and fields, which can be used for manual validation|
|[sample_differentiate_output_models_trained_with_and_without_labels.py][sample_differentiate_output_models_trained_with_and_without_labels] and [sample_differentiate_output_models_trained_with_and_without_labels_async.py][sample_differentiate_output_models_trained_with_and_without_labels_async]|See the differences in output when using a custom model trained with labeled data and one trained with unlabeled data|

Expand Down Expand Up @@ -94,4 +95,6 @@ what you can do with the Azure Form Recognizer client library.
[sample_train_model_without_labels]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/sample_train_model_without_labels.py
[sample_train_model_without_labels_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/async_samples/sample_train_model_without_labels_async.py
[sample_copy_model]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/sample_copy_model.py
[sample_copy_model_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/async_samples/sample_copy_model_async.py
[sample_copy_model_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/async_samples/sample_copy_model_async.py
[sample_strongly_typing_recognized_form]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/sample_strongly_typing_recognized_form.py
[sample_strongly_typing_recognized_form_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/async_samples/sample_strongly_typing_recognized_form_async.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,12 @@
FILE: sample_recognize_receipts_async.py
DESCRIPTION:
This sample demonstrates how to recognize US sales receipts from a file.
This sample demonstrates how to recognize and extract common fields from US receipts,
using a pre-trained receipt model. For a suggested approach to extracting information
from receipts, see sample_strongly_typed_recognized_form_async.py.
See fields found on a receipt here:
https://aka.ms/azsdk/python/formrecognizer/receiptfields
USAGE:
python sample_recognize_receipts_async.py
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,12 @@
FILE: sample_recognize_receipts_from_url_async.py
DESCRIPTION:
This sample demonstrates how to recognize US sales receipts from a URL.
This sample demonstrates how to recognize and extract common fields from a US receipt URL,
using a pre-trained receipt model. For a suggested approach to extracting information
from receipts, see sample_strongly_typed_recognized_form_async.py.
See fields found on a receipt here:
https://aka.ms/azsdk/python/formrecognizer/receiptfields
USAGE:
python sample_recognize_receipts_from_url_async.py
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# coding: utf-8

# -------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for
# license information.
# --------------------------------------------------------------------------

"""
FILE: sample_strongly_typed_recognized_form_async.py
DESCRIPTION:
This sample demonstrates how to use the fields in your recognized forms to create an object with
strongly-typed fields. The pre-trained receipt method will be used to illustrate this sample, but
note that a similar approach can be used for any custom form as long as you properly update the
fields' names and types.
See fields found on a receipt here:
https://aka.ms/azsdk/python/formrecognizer/receiptfields
USAGE:
python sample_strongly_typed_recognized_form_async.py
Set the environment variables with your own values before running the sample:
1) AZURE_FORM_RECOGNIZER_ENDPOINT - the endpoint to your Cognitive Services resource.
2) AZURE_FORM_RECOGNIZER_KEY - your Form Recognizer API key
"""

import os
import asyncio
from azure.ai.formrecognizer import FormField


class Receipt(object):
"""Creates a strongly-typed Receipt class from the fields returned in a RecognizedForm.
If a specific field is not found on the receipt, it will return None.
See fields found on a receipt here:
https://aka.ms/azsdk/python/formrecognizer/receiptfields
"""

def __init__(self, form):
self.receipt_type = form.fields.get("ReceiptType", FormField())
self.merchant_name = form.fields.get("MerchantName", FormField())
self.merchant_address = form.fields.get("MerchantAddress", FormField())
self.merchant_phone_number = form.fields.get("MerchantPhoneNumber", FormField())
self.receipt_items = self.convert_to_receipt_item(form.fields.get("Items", FormField()))
self.subtotal = form.fields.get("Subtotal", FormField())
self.tax = form.fields.get("Tax", FormField())
self.tip = form.fields.get("Tip", FormField())
self.total = form.fields.get("Total", FormField())
self.transaction_date = form.fields.get("TransactionDate", FormField())
self.transaction_time = form.fields.get("TransactionTime", FormField())

def convert_to_receipt_item(self, items):
"""Converts Items in a receipt to a list of strongly-typed ReceiptItem
"""
if items is None:
return []
return [ReceiptItem(item) for item in items.value]


class ReceiptItem(object):
"""Creates a strongly-typed ReceiptItem for every receipt item found in a RecognizedForm
"""

def __init__(self, item):
self.name = item.value.get("Name", FormField())
self.quantity = item.value.get("Quantity", FormField())
self.price = item.value.get("Price", FormField())
self.total_price = item.value.get("TotalPrice", FormField())


class StronglyTypedRecognizedFormSampleAsync(object):

async def strongly_typed_receipt_async(self):
path_to_sample_forms = os.path.abspath(os.path.join(os.path.abspath(__file__), "..", "..", "./sample_forms/receipt/contoso-allinone.jpg"))

from azure.core.credentials import AzureKeyCredential
from azure.ai.formrecognizer.aio import FormRecognizerClient

endpoint = os.environ["AZURE_FORM_RECOGNIZER_ENDPOINT"]
key = os.environ["AZURE_FORM_RECOGNIZER_KEY"]

async with FormRecognizerClient(
endpoint=endpoint, credential=AzureKeyCredential(key)
) as form_recognizer_client:

with open(path_to_sample_forms, "rb") as f:
poller = await form_recognizer_client.begin_recognize_receipts(receipt=f)
receipts = await poller.result()

for receipt in receipts:
receipt = Receipt(receipt)
print("Receipt Type: {} has confidence: {}".format(receipt.receipt_type.value, receipt.receipt_type.confidence))
print("Merchant Name: {} has confidence: {}".format(receipt.merchant_name.value, receipt.merchant_name.confidence))
print("Transaction Date: {} has confidence: {}".format(receipt.transaction_date.value, receipt.transaction_date.confidence))
print("Receipt items:")
for item in receipt.receipt_items:
print("...Item Name: {} has confidence: {}".format(item.name.value, item.name.confidence))
print("...Item Quantity: {} has confidence: {}".format(item.quantity.value, item.quantity.confidence))
print("...Individual Item Price: {} has confidence: {}".format(item.price.value, item.price.confidence))
print("...Total Item Price: {} has confidence: {}".format(item.total_price.value, item.total_price.confidence))
print("Subtotal: {} has confidence: {}".format(receipt.subtotal.value, receipt.subtotal.confidence))
print("Tax: {} has confidence: {}".format(receipt.tax.value, receipt.tax.confidence))
print("Tip: {} has confidence: {}".format(receipt.tip.value, receipt.tip.confidence))
print("Total: {} has confidence: {}".format(receipt.total.value, receipt.total.confidence))


async def main():
sample = StronglyTypedRecognizedFormSampleAsync()
await sample.strongly_typed_receipt_async()


if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,12 @@
FILE: sample_recognize_receipts.py
DESCRIPTION:
This sample demonstrates how to recognize US sales receipts from a file.
This sample demonstrates how to recognize and extract common fields from US receipts,
using a pre-trained receipt model. For a suggested approach to extracting information
from receipts, see sample_strongly_typed_recognized_form.py.
See fields found on a receipt here:
https://aka.ms/azsdk/python/formrecognizer/receiptfields
USAGE:
python sample_recognize_receipts.py
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,12 @@
FILE: sample_recognize_receipts_from_url.py
DESCRIPTION:
This sample demonstrates how to recognize US sales receipts from a URL.
This sample demonstrates how to recognize and extract common fields from a US receipt URL,
using a pre-trained receipt model. For a suggested approach to extracting information
from receipts, see sample_strongly_typed_recognized_form.py.
See fields found on a receipt here:
https://aka.ms/azsdk/python/formrecognizer/receiptfields
USAGE:
python sample_recognize_receipts_from_url.py
Expand Down
Loading

0 comments on commit c3a2d16

Please sign in to comment.