Skip to content

Commit

Permalink
[formrecognizer] support Copy API (#11372)
Browse files Browse the repository at this point in the history
* add copy feature gen code

* add copy implementation (wip)

* update copy implementation

* add copy model samples

* add back get copy auth method and update samples

* add CopyAuthorization to init

* changes from feedback on design

* raise better error messages in polling for failed copying

* update readme/changelog

* updating sample snippets

* renames to align with .net

* change copy target to dict

* add copy sync/async tests

* make mypy happy

* update docstrings

* review feedback

* rename authorize_copy_target -> get_copy_authorization

* feedback + add test for copy authorization

* fix to testcase

* change description in samples

* hardcode region

* construct resource id in testcase
  • Loading branch information
kristapratico authored May 20, 2020
1 parent 0667e82 commit 8f5fb10
Show file tree
Hide file tree
Showing 37 changed files with 3,238 additions and 67 deletions.
1 change: 1 addition & 0 deletions sdk/formrecognizer/azure-ai-formrecognizer/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ For recognize receipt methods, parameters have been renamed to `receipt` and `re

**New features**

- Support to copy a custom model from one Form Recognizer resource to another
- Authentication using `azure-identity` credentials now supported
- see the [Azure Identity documentation](https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/identity/azure-identity/README.md) for more information

Expand Down
4 changes: 4 additions & 0 deletions sdk/formrecognizer/azure-ai-formrecognizer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ See the full details regarding [authentication][cognitive_authentication] of cog
- Training custom models to recognize all fields and values found in your custom forms. A `CustomFormModel` is returned indicating the form types the model will recognize, and the fields it will extract for each form type. See the [service's documents][fr-train-without-labels] for a more detailed explanation.
- Training custom models to recognize specific fields and values you specify by labeling your custom forms. A `CustomFormModel` is returned indicating the fields the model will extract, as well as the estimated accuracy for each field. See the [service's documents][fr-train-with-labels] for a more detailed explanation.
- Managing models created in your account.
- Copying a custom model from one Form Recognizer resource to another.

Please note that models can also be trained using a graphical user interface such as the [Form Recognizer Labeling Tool][fr-labeling-tool].

Expand Down Expand Up @@ -389,6 +390,7 @@ with Form Recognizer and require Python 3.5 or later.
* Train a model without labels: [sample_train_model_without_labels.py][sample_train_model_without_labels] ([async version][sample_train_model_without_labels_async])
* Train a model with labels: [sample_train_model_with_labels.py][sample_train_model_with_labels] ([async version][sample_train_model_with_labels_async])
* Manage custom models: [sample_manage_custom_models.py][sample_manage_custom_models] ([async_version][sample_manage_custom_models_async])
* Copy a model between Form Recognizer resources: [sample_copy_model.py][sample_copy_model] ([async_version][sample_copy_model_async])

### Additional documentation

Expand Down Expand Up @@ -459,3 +461,5 @@ This project has adopted the [Microsoft Open Source Code of Conduct][code_of_con
[sample_train_model_with_labels_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/async_samples/sample_train_model_with_labels_async.py
[sample_train_model_without_labels]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/sample_train_model_without_labels.py
[sample_train_model_without_labels_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/async_samples/sample_train_model_without_labels_async.py
[sample_copy_model]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/sample_copy_model.py
[sample_copy_model_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/async_samples/sample_copy_model_async.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,30 +6,39 @@

# pylint: disable=protected-access

import json
from typing import (
Optional,
Any,
Iterable,
Dict,
Union,
TYPE_CHECKING,
)
from azure.core.tracing.decorator import distributed_trace
from azure.core.polling import LROPoller
from azure.core.polling.base_polling import LROBasePolling
from ._generated.models import Model
from ._generated._form_recognizer_client import FormRecognizerClient as FormRecognizer
from ._generated.models import TrainRequest, TrainSourceFilter
from ._generated.models import (
TrainRequest,
TrainSourceFilter,
CopyRequest,
Model,
CopyOperationResult,
CopyAuthorizationResult
)
from ._helpers import error_map, get_authentication_policy, POLLING_INTERVAL
from ._models import (
CustomFormModelInfo,
AccountProperties,
CustomFormModel
)
from ._polling import TrainingPolling
from ._polling import TrainingPolling, CopyPolling
from ._user_agent import USER_AGENT
from ._form_recognizer_client import FormRecognizerClient
if TYPE_CHECKING:
from azure.core.credentials import AzureKeyCredential, TokenCredential
from azure.core.pipeline import PipelineResponse
from azure.core.pipeline.transport import HttpResponse
PipelineResponseType = HttpResponse

Expand Down Expand Up @@ -239,6 +248,99 @@ def get_custom_model(self, model_id, **kwargs):
response = self._client.get_custom_model(model_id=model_id, include_keys=True, error_map=error_map, **kwargs)
return CustomFormModel._from_generated(response)

@distributed_trace
def get_copy_authorization(self, resource_id, resource_region, **kwargs):
# type: (str, str, Any) -> Dict[str, Union[str, int]]
"""Generate authorization for copying a custom model into the target Form Recognizer resource.
This should be called by the target resource (where the model will be copied to)
and the output can be passed as the `target` parameter into :func:`~begin_copy_model()`.
:param str resource_id: Azure Resource Id of the target Form Recognizer resource
where the model will be copied to.
:param str resource_region: Location of the target Form Recognizer resource. A valid Azure
region name supported by Cognitive Services.
:return: A dictionary with values for the copy authorization -
"modelId", "accessToken", "resourceId", "resourceRegion", and "expirationDateTimeTicks".
:rtype: Dict[str, Union[str, int]]
:raises ~azure.core.exceptions.HttpResponseError:
.. admonition:: Example:
.. literalinclude:: ../samples/sample_copy_model.py
:start-after: [START get_copy_authorization]
:end-before: [END get_copy_authorization]
:language: python
:dedent: 8
:caption: Authorize the target resource to receive the copied model
"""

response = self._client.generate_model_copy_authorization( # type: ignore
cls=lambda pipeline_response, deserialized, response_headers: pipeline_response,
error_map=error_map,
**kwargs
) # type: PipelineResponse
target = json.loads(response.http_response.text())
target["resourceId"] = resource_id
target["resourceRegion"] = resource_region
return target

@distributed_trace
def begin_copy_model(
self,
model_id, # type: str
target, # type: Dict
**kwargs # type: Any
):
# type: (...) -> LROPoller
"""Copy a custom model stored in this resource (the source) to the user specified
target Form Recognizer resource. This should be called with the source Form Recognizer resource
(with the model that is intended to be copied). The `target` parameter should be supplied from the
target resource's output from calling the :func:`~get_copy_authorization()` method.
:param str model_id: Model identifier of the model to copy to target resource.
:param dict target:
The copy authorization generated from the target resource's call to
:func:`~get_copy_authorization()`.
:keyword int polling_interval: Default waiting time between two polls for LRO operations if
no Retry-After header is present.
:return: An instance of an LROPoller. Call `result()` on the poller
object to return a :class:`~azure.ai.formrecognizer.CustomFormModelInfo`.
:rtype: ~azure.core.polling.LROPoller[~azure.ai.formrecognizer.CustomFormModelInfo]
:raises ~azure.core.exceptions.HttpResponseError:
.. admonition:: Example:
.. literalinclude:: ../samples/sample_copy_model.py
:start-after: [START begin_copy_model]
:end-before: [END begin_copy_model]
:language: python
:dedent: 8
:caption: Copy a model from the source resource to the target resource
"""

polling_interval = kwargs.pop("polling_interval", POLLING_INTERVAL)

def _copy_callback(raw_response, _, headers): # pylint: disable=unused-argument
copy_result = self._client._deserialize(CopyOperationResult, raw_response)
return CustomFormModelInfo._from_generated(copy_result, target["modelId"])

return self._client.begin_copy_custom_model( # type: ignore
model_id=model_id,
copy_request=CopyRequest(
target_resource_id=target["resourceId"],
target_resource_region=target["resourceRegion"],
copy_authorization=CopyAuthorizationResult(
access_token=target["accessToken"],
model_id=target["modelId"],
expiration_date_time_ticks=target["expirationDateTimeTicks"]
)
),
cls=kwargs.pop("cls", _copy_callback),
polling=LROBasePolling(timeout=polling_interval, lro_algorithms=[CopyPolling()], **kwargs),
error_map=error_map,
**kwargs
)

def get_form_recognizer_client(self, **kwargs):
# type: (Any) -> FormRecognizerClient
"""Get an instance of a FormRecognizerClient from FormTrainingClient.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

Expand Down Expand Up @@ -46,6 +46,7 @@ def __init__(
self.credential = credential
self.endpoint = endpoint
self.credential_scopes = ['https://cognitiveservices.azure.com/.default']
self.credential_scopes.extend(kwargs.pop('credential_scopes', []))
kwargs.setdefault('sdk_moniker', 'ai-formrecognizer/{}'.format(VERSION))
self._configure(**kwargs)

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

VERSION = "1.0.0b1"
VERSION = "1.0.0b2"
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

Expand Down Expand Up @@ -43,6 +43,7 @@ def __init__(
self.credential = credential
self.endpoint = endpoint
self.credential_scopes = ['https://cognitiveservices.azure.com/.default']
self.credential_scopes.extend(kwargs.pop('credential_scopes', []))
kwargs.setdefault('sdk_moniker', 'ai-formrecognizer/{}'.format(VERSION))
self._configure(**kwargs)

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

Expand Down
Loading

0 comments on commit 8f5fb10

Please sign in to comment.