Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[formrecognizer] support Copy API #11372

Merged
merged 26 commits into from
May 20, 2020
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
f73471a
add copy feature gen code
kristapratico May 5, 2020
3903319
add copy implementation (wip)
kristapratico May 5, 2020
d7160bb
update copy implementation
kristapratico May 5, 2020
24a1636
add copy model samples
kristapratico May 5, 2020
957cce1
add back get copy auth method and update samples
kristapratico May 11, 2020
3ff9b0b
add CopyAuthorization to init
kristapratico May 11, 2020
d127002
changes from feedback on design
kristapratico May 12, 2020
a8ac624
raise better error messages in polling for failed copying
kristapratico May 12, 2020
19c77c7
update readme/changelog
kristapratico May 12, 2020
40ac006
updating sample snippets
kristapratico May 12, 2020
967cff4
Merge branch 'master' into fr-copy-api
kristapratico May 12, 2020
464877f
renames to align with .net
kristapratico May 12, 2020
ff525cb
change copy target to dict
kristapratico May 13, 2020
d65ab3e
add copy sync/async tests
kristapratico May 14, 2020
cb25fc7
merge master
kristapratico May 14, 2020
23b1d3a
make mypy happy
kristapratico May 14, 2020
ed4d8e3
update docstrings
kristapratico May 14, 2020
0e689f8
merge master
kristapratico May 14, 2020
bce5031
Merge branch 'master' into fr-copy-api
kristapratico May 18, 2020
a2a5dac
review feedback
kristapratico May 18, 2020
4736418
rename authorize_copy_target -> get_copy_authorization
kristapratico May 18, 2020
ff26bc6
feedback + add test for copy authorization
kristapratico May 19, 2020
ca64271
fix to testcase
kristapratico May 20, 2020
c100972
change description in samples
kristapratico May 20, 2020
688c3eb
hardcode region
kristapratico May 20, 2020
58a2210
construct resource id in testcase
kristapratico May 20, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions sdk/formrecognizer/azure-ai-formrecognizer/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

**New features**

- Support to copy a custom model from one Form Recognizer resource to another
- Authentication using `azure-identity` credentials now supported
- see the [Azure Identity documentation](https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/identity/azure-identity/README.md) for more information

Expand Down
4 changes: 4 additions & 0 deletions sdk/formrecognizer/azure-ai-formrecognizer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ See the full details regarding [authentication][cognitive_authentication] of cog
- Training custom models to recognize all fields and values found in your custom forms. A `CustomFormModel` is returned indicating the form types the model will recognize, and the fields it will extract for each form type. See the [service's documents][fr-train-without-labels] for a more detailed explanation.
- Training custom models to recognize specific fields and values you specify by labeling your custom forms. A `CustomFormModel` is returned indicating the fields the model will extract, as well as the estimated accuracy for each field. See the [service's documents][fr-train-with-labels] for a more detailed explanation.
- Managing models created in your account.
- Copying a custom model from one Form Recognizer resource to another.

Please note that models can also be trained using a graphical user interface such as the [Form Recognizer Labeling Tool][fr-labeling-tool].

Expand Down Expand Up @@ -389,6 +390,7 @@ with Form Recognizer and require Python 3.5 or later.
* Train a model without labels: [sample_train_model_without_labels.py][sample_train_model_without_labels] ([async version][sample_train_model_without_labels_async])
* Train a model with labels: [sample_train_model_with_labels.py][sample_train_model_with_labels] ([async version][sample_train_model_with_labels_async])
* Manage custom models: [sample_manage_custom_models.py][sample_manage_custom_models] ([async_version][sample_manage_custom_models_async])
* Copy a model between Form Recognizer resources: [sample_copy_model.py][sample_copy_model] ([async_version][sample_copy_model_async])

### Additional documentation

Expand Down Expand Up @@ -459,3 +461,5 @@ This project has adopted the [Microsoft Open Source Code of Conduct][code_of_con
[sample_train_model_with_labels_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/async_samples/sample_train_model_with_labels_async.py
[sample_train_model_without_labels]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/sample_train_model_without_labels.py
[sample_train_model_without_labels_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/async_samples/sample_train_model_without_labels_async.py
[sample_copy_model]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/sample_copy_model.py
[sample_copy_model_async]: https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/formrecognizer/azure-ai-formrecognizer/samples/async_samples/sample_copy_model_async.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,30 +6,39 @@

# pylint: disable=protected-access

import json
from typing import (
Optional,
Any,
Iterable,
Dict,
Union,
TYPE_CHECKING,
)
from azure.core.tracing.decorator import distributed_trace
from azure.core.polling import LROPoller
from azure.core.polling.base_polling import LROBasePolling
from ._generated.models import Model
from ._generated._form_recognizer_client import FormRecognizerClient as FormRecognizer
from ._generated.models import TrainRequest, TrainSourceFilter
from ._generated.models import (
TrainRequest,
TrainSourceFilter,
CopyRequest,
Model,
CopyOperationResult,
CopyAuthorizationResult
)
from ._helpers import error_map, get_authentication_policy, POLLING_INTERVAL
from ._models import (
CustomFormModelInfo,
AccountProperties,
CustomFormModel
)
from ._polling import TrainingPolling
from ._polling import TrainingPolling, CopyPolling
from ._user_agent import USER_AGENT
from ._form_recognizer_client import FormRecognizerClient
if TYPE_CHECKING:
from azure.core.credentials import AzureKeyCredential, TokenCredential
from azure.core.pipeline import PipelineResponse
from azure.core.pipeline.transport import HttpResponse
PipelineResponseType = HttpResponse

Expand Down Expand Up @@ -239,6 +248,99 @@ def get_custom_model(self, model_id, **kwargs):
response = self._client.get_custom_model(model_id=model_id, include_keys=True, error_map=error_map, **kwargs)
return CustomFormModel._from_generated(response)

@distributed_trace
def get_copy_authorization(self, resource_id, resource_region, **kwargs):
# type: (str, str, Any) -> Dict[str, Union[str, int]]
"""Generate authorization for copying a custom model into the target Form Recognizer resource.
This should be called by the target resource (where the model will be copied to)
and the output can be passed as the `target` parameter into :func:`~begin_copy_model()`.

:param str resource_id: Azure Resource Id of the target Form Recognizer resource
where the model will be copied to.
:param str resource_region: Location of the target Form Recognizer resource. A valid Azure
region name supported by Cognitive Services.
:return: A dictionary with values for the copy authorization -
"modelId", "accessToken", "resourceId", "resourceRegion", and "expirationDateTimeTicks".
:rtype: Dict[str, Union[str, int]]
:raises ~azure.core.exceptions.HttpResponseError:

.. admonition:: Example:

.. literalinclude:: ../samples/sample_copy_model.py
:start-after: [START get_copy_authorization]
:end-before: [END get_copy_authorization]
:language: python
:dedent: 8
:caption: Authorize the target resource to receive the copied model
"""

response = self._client.generate_model_copy_authorization( # type: ignore
cls=lambda pipeline_response, deserialized, response_headers: pipeline_response,
error_map=error_map,
**kwargs
) # type: PipelineResponse
target = json.loads(response.http_response.text())
target["resourceId"] = resource_id
target["resourceRegion"] = resource_region
return target

@distributed_trace
def begin_copy_model(
self,
model_id, # type: str
target, # type: Dict
**kwargs # type: Any
):
# type: (...) -> LROPoller
"""Copy a custom model stored in this resource (the source) to the user specified
target Form Recognizer resource. This should be called with the source Form Recognizer resource
(with the model that is intended to be copied). The `target` parameter should be supplied from the
target resource's output from calling the :func:`~get_copy_authorization()` method.

:param str model_id: Model identifier of the model to copy to target resource.
:param dict target:
The copy authorization generated from the target resource's call to
:func:`~get_copy_authorization()`.
:keyword int polling_interval: Default waiting time between two polls for LRO operations if
no Retry-After header is present.
:return: An instance of an LROPoller. Call `result()` on the poller
object to return a :class:`~azure.ai.formrecognizer.CustomFormModelInfo`.
:rtype: ~azure.core.polling.LROPoller[~azure.ai.formrecognizer.CustomFormModelInfo]
:raises ~azure.core.exceptions.HttpResponseError:

.. admonition:: Example:

.. literalinclude:: ../samples/sample_copy_model.py
:start-after: [START begin_copy_model]
:end-before: [END begin_copy_model]
:language: python
:dedent: 8
:caption: Copy a model from the source resource to the target resource
"""

polling_interval = kwargs.pop("polling_interval", POLLING_INTERVAL)

def _copy_callback(raw_response, _, headers): # pylint: disable=unused-argument
kristapratico marked this conversation as resolved.
Show resolved Hide resolved
copy_result = self._client._deserialize(CopyOperationResult, raw_response)
return CustomFormModelInfo._from_generated(copy_result, target["modelId"])

return self._client.begin_copy_custom_model( # type: ignore
model_id=model_id,
copy_request=CopyRequest(
target_resource_id=target["resourceId"],
target_resource_region=target["resourceRegion"],
copy_authorization=CopyAuthorizationResult(
access_token=target["accessToken"],
model_id=target["modelId"],
expiration_date_time_ticks=target["expirationDateTimeTicks"]
kristapratico marked this conversation as resolved.
Show resolved Hide resolved
)
),
cls=kwargs.pop("cls", _copy_callback),
polling=LROBasePolling(timeout=polling_interval, lro_algorithms=[CopyPolling()], **kwargs),
error_map=error_map,
**kwargs
)

def get_form_recognizer_client(self, **kwargs):
# type: (Any) -> FormRecognizerClient
"""Get an instance of a FormRecognizerClient from FormTrainingClient.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

Expand Down Expand Up @@ -46,6 +46,7 @@ def __init__(
self.credential = credential
self.endpoint = endpoint
self.credential_scopes = ['https://cognitiveservices.azure.com/.default']
self.credential_scopes.extend(kwargs.pop('credential_scopes', []))
kwargs.setdefault('sdk_moniker', 'ai-formrecognizer/{}'.format(VERSION))
self._configure(**kwargs)

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

VERSION = "1.0.0b1"
VERSION = "1.0.0b2"
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

Expand Down Expand Up @@ -43,6 +43,7 @@ def __init__(
self.credential = credential
self.endpoint = endpoint
self.credential_scopes = ['https://cognitiveservices.azure.com/.default']
self.credential_scopes.extend(kwargs.pop('credential_scopes', []))
kwargs.setdefault('sdk_moniker', 'ai-formrecognizer/{}'.format(VERSION))
self._configure(**kwargs)

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# coding=utf-8
# --------------------------------------------------------------------------
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6246, generator: {generator})
# Code generated by Microsoft (R) AutoRest Code Generator (autorest: 3.0.6282, generator: {generator})
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

Expand Down
Loading