[TA] Merge branch feature/textanalytics/custom (#24941)

* [TextAnalytics] Generated client from 3.2-preview.2 swagger (#23536) * [TA] Added SingleCategoryClassify functionality (#24235) * [TA] Added MultiCategoryClassify Functionality (#24237) * [TA] Added RecognizeCustomEntities Functionality (#24245) * [TA] Expose ActionName and enable multiple actions from same type (#24619) * Rerecorded all tests excluding AAD ones (#24913) * re-record AAD tests (#24919) * [TA] Enable CI for live tests for custom features (#24916) * add comments Co-authored-by: Caio Saldanha <camaiaor@microsoft.com> Co-authored-by: Ahmed Leithy <v-aleithy@microsoft.com> Co-authored-by: Salah Mostafa <zulamostafa@gmail.com> Co-authored-by: Salah Mostafa <v-samostafa@microsoft.com>
Azure · Oct 26, 2021 · 7cc705d · 7cc705d
1 parent 4a5efb7
commit 7cc705d
Show file tree

Hide file tree

Showing 412 changed files with 38,204 additions and 57,887 deletions.
diff --git a/sdk/textanalytics/Azure.AI.TextAnalytics/CHANGELOG.md b/sdk/textanalytics/Azure.AI.TextAnalytics/CHANGELOG.md
@@ -3,13 +3,18 @@
 ## 5.2.0-beta.2 (Unreleased)
 
 ### Features Added
+- Adding support for three new actions in `StartAnalyzeActions`: `RecognizeCustomEntities`, `SingleCategoryClassify`, and `MultiCategoriesClassify`. The new actions allow you to use custom models to perform entity recognition and category classification.
+- Added property `ActionName` to all `xxActions` input types so user can specify a name per action. If not provided, service will generate a name.
+- Added property `ActionName` to all `xxActionResult` output types that displays the name of each action.
+- Added suppport for multiple actions of the same type.
 
 ### Breaking Changes
 
 ### Bugs Fixed
 - `AnalyzeActionsOperation.GetValuesAsync()` and `AnalyzeHealthcareEntitiesOperation.GetValuesAsync()` are now validating that the operation has completed successfully before attempting to return any values. An `InvalidOperationException` is thrown if this is not true.
 
 ### Other Changes
+- We are now targeting the service version `3.2-preview.2` API as the default instead of `3.2-preview.1`.
 
 ## 5.2.0-beta.1 (2021-08-09)
 

diff --git a/sdk/textanalytics/Azure.AI.TextAnalytics/README.md b/sdk/textanalytics/Azure.AI.TextAnalytics/README.md
@@ -600,6 +600,7 @@ This functionality allows running multiple actions in one or more documents. Act
         int docNumber = 1;
         foreach (RecognizeEntitiesActionResult entitiesActionResults in entitiesResults)
         {
+            Console.WriteLine($" Action name: {entitiesActionResults.ActionName}");
             foreach (RecognizeEntitiesResult documentResults in entitiesActionResults.DocumentsResults)
             {
                 Console.WriteLine($" Document #{docNumber++}");

diff --git a/sdk/textanalytics/Azure.AI.TextAnalytics/api/Azure.AI.TextAnalytics.netstandard2.0.cs b/sdk/textanalytics/Azure.AI.TextAnalytics/api/Azure.AI.TextAnalytics.netstandard2.0.cs
diff --git a/...textanalytics/Azure.AI.TextAnalytics/samples/Sample10_SingleCategoryClassify.md b/...textanalytics/Azure.AI.TextAnalytics/samples/Sample10_SingleCategoryClassify.md
@@ -0,0 +1,91 @@
+# Perform Custom Single Category Classification on Documents
+This sample demonstrates how to run a single category classification action in one or more documents. To get started you will need a Text Analytics endpoint and credentials. See [README][README] for links and instructions.
+
+## Creating a `TextAnalyticsClient`
+
+To create a new `TextAnalyticsClient` to perform a single category classify on a document, you need a Text Analytics endpoint and credentials.  You can use the [DefaultAzureCredential][DefaultAzureCredential] to try a number of common authentication methods optimized for both running as a service and development.  In the sample below, however, you'll use a Text Analytics API key credential by creating an `AzureKeyCredential` object, that if needed, will allow you to update the API key without creating a new client.
+
+You can set `endpoint` and `apiKey` based on an environment variable, a configuration setting, or any way that works for your application.
+
+```C# Snippet:CreateTextAnalyticsClient
+string endpoint = "<endpoint>";
+string apiKey = "<apiKey>";
+var client = new TextAnalyticsClient(new Uri(endpoint), new AzureKeyCredential(apiKey));
+```
+
+## Performing Single Category Classify on one or multiple documents
+
+To perform Custom Single Category Classification in one or multiple documents, set up a `SingleCategoryClassifyAction` and call `StartAnalyzeActionsAsync` on the documents. The result is a Long Running Operation of type `AnalyzeActionsOperation` which polls for the results from the API.
+
+```C# Snippet:TextAnalyticsSingleCategoryClassifyAsync
+// Get input document.
+string document = @"I need a reservation for an indoor restaurant in China. Please don't stop the music. Play music and add it to my playlist.";
+
+// Prepare analyze operation input. You can add multiple documents to this list and perform the same
+// operation to all of them.
+var batchInput = new List<string>
+{
+    document
+};
+
+// Set project and deployment names of the target model
+string projectName = "<projectName>";
+string deploymentName = "<deploymentName>";
+
+var singleCategoryClassifyAction = new SingleCategoryClassifyAction(projectName, deploymentName);
+
+TextAnalyticsActions actions = new TextAnalyticsActions()
+{
+    SingleCategoryClassifyActions = new List<SingleCategoryClassifyAction>() { singleCategoryClassifyAction }
+};
+
+// Start analysis process.
+AnalyzeActionsOperation operation = await client.StartAnalyzeActionsAsync(batchInput, actions);
+
+await operation.WaitForCompletionAsync();
+```
+
+The returned `AnalyzeActionsOperation` contains general information about the status of the operation. It can be requested while the operation is running or when it has completed. For example:
+
+```C# Snippet:TextAnalyticsSingleCategoryClassifyOperationStatus
+// View operation status.
+Console.WriteLine($"AnalyzeActions operation has completed");
+Console.WriteLine();
+
+Console.WriteLine($"Created On   : {operation.CreatedOn}");
+Console.WriteLine($"Expires On   : {operation.ExpiresOn}");
+Console.WriteLine($"Id           : {operation.Id}");
+Console.WriteLine($"Status       : {operation.Status}");
+Console.WriteLine($"Last Modified: {operation.LastModified}");
+Console.WriteLine();
+```
+
+To view the final results of the long-running operation:
+
+```C# Snippet:TextAnalyticsSingleCategoryClassifyAsyncViewResults
+// View operation results.
+await foreach (AnalyzeActionsResult documentsInPage in operation.Value)
+{
+    IReadOnlyCollection<SingleCategoryClassifyActionResult> singleClassificationActionResults = documentsInPage.SingleCategoryClassifyResults;
+
+    foreach (SingleCategoryClassifyActionResult classificationActionResults in singleClassificationActionResults)
+    {
+        Console.WriteLine($" Action name: {classificationActionResults.ActionName}");
+        foreach (SingleCategoryClassifyResult documentResults in classificationActionResults.DocumentsResults)
+        {
+            Console.WriteLine($"  Class category \"{documentResults.Classification.Category}\" predicted with a confidence score of {documentResults.Classification.ConfidenceScore}.");
+            Console.WriteLine();
+        }
+    }
+}
+```
+
+To see the full example source files, see:
+
+* [Synchronously SingleCategoryClassify](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample10_SingleCategoryClassify.cs)
+* [Asynchronously SingleCategoryClassify](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample10_SingleCategoryClassifyAsync.cs)
+* [Synchronously SingleCategoryClassify Convenience](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample10_SingleCategoryClassifyConvenience.cs)
+* [Asynchronously SingleCategoryClassify Convenience](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample10_SingleCategoryClassifyConvenienceAsync.cs)
+
+[DefaultAzureCredential]: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/identity/Azure.Identity/README.md
+[README]: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/README.md
diff --git a/...xtanalytics/Azure.AI.TextAnalytics/samples/Sample11_ClassifyCustomCategories.md b/...xtanalytics/Azure.AI.TextAnalytics/samples/Sample11_ClassifyCustomCategories.md
@@ -0,0 +1,100 @@
+# Perform Custom Multiple Category Classification in Documents
+This sample demonstrates how to run a Multi Category Classification action in one or more documents. To get started you will need a Text Analytics endpoint and credentials. See [README][README] for links and instructions.
+
+## Creating a `TextAnalyticsClient`
+
+To create a new `TextAnalyticsClient` to perform a custom multi category classification on a document, you need a Text Analytics endpoint and credentials.  You can use the [DefaultAzureCredential][DefaultAzureCredential] to try a number of common authentication methods optimized for both running as a service and development.  In the sample below, however, you'll use a Text Analytics API key credential by creating an `AzureKeyCredential` object, that if needed, will allow you to update the API key without creating a new client.
+
+You can set `endpoint` and `apiKey` based on an environment variable, a configuration setting, or any way that works for your application.
+
+```C# Snippet:CreateTextAnalyticsClient
+string endpoint = "<endpoint>";
+string apiKey = "<apiKey>";
+var client = new TextAnalyticsClient(new Uri(endpoint), new AzureKeyCredential(apiKey));
+```
+
+## Performing Custom Multiple Category Classification in one or multiple documents
+
+To perform Multiple Category Classification in one or multiple documents, set up a `MultiCategoryClassifyAction` and call `StartAnalyzeActionsAsync` on the documents. The result is a Long Running Operation of type `AnalyzeActionsOperation` which polls for the results from the API.
+
+```C# Snippet:TextAnalyticsMultiCategoryClassifyAsync
+// Get input document.
+string document = @"I need a reservation for an indoor restaurant in China. Please don't stop the music. Play music and add it to my playlist.";
+
+// Prepare analyze operation input. You can add multiple documents to this list and perform the same
+// operation to all of them.
+var batchInput = new List<string>
+{
+    document
+};
+
+// Set project and deployment names of the target model
+string projectName = "<projectName>";
+string deploymentName = "<deploymentName>";
+
+var multiCategoryClassifyAction = new MultiCategoryClassifyAction(projectName, deploymentName);
+
+TextAnalyticsActions actions = new TextAnalyticsActions()
+{
+    MultiCategoryClassifyActions = new List<MultiCategoryClassifyAction>() { multiCategoryClassifyAction }
+};
+
+// Start analysis process.
+AnalyzeActionsOperation operation = await client.StartAnalyzeActionsAsync(batchInput, actions);
+
+await operation.WaitForCompletionAsync();
+```
+
+The returned `AnalyzeActionsOperation` contains general information about the status of the operation. It can be requested while the operation is running or when it has completed. For example:
+
+```C# Snippet:TextAnalyticsMultiCategoryClassifyOperationStatus
+// View operation status.
+Console.WriteLine($"AnalyzeActions operation has completed");
+Console.WriteLine();
+
+Console.WriteLine($"Created On   : {operation.CreatedOn}");
+Console.WriteLine($"Expires On   : {operation.ExpiresOn}");
+Console.WriteLine($"Id           : {operation.Id}");
+Console.WriteLine($"Status       : {operation.Status}");
+Console.WriteLine($"Last Modified: {operation.LastModified}");
+Console.WriteLine();
+```
+
+To view the final results of the long-running operation:
+
+```C# Snippet:TextAnalyticsMultiCategoryClassifyAsyncViewResults
+// View operation results.
+await foreach (AnalyzeActionsResult documentsInPage in operation.Value)
+{
+    IReadOnlyCollection<MultiCategoryClassifyActionResult> multiClassificationActionResults = documentsInPage.MultiCategoryClassifyResults;
+
+    foreach (MultiCategoryClassifyActionResult classificationActionResults in multiClassificationActionResults)
+    {
+        Console.WriteLine($" Action name: {classificationActionResults.ActionName}");
+        foreach (MultiCategoryClassifyResult documentResults in classificationActionResults.DocumentsResults)
+        {
+            if (documentResults.Classifications.Count > 0)
+            {
+                Console.WriteLine($"  The following classes were predicted for this document:");
+
+                foreach (ClassificationCategory classification in documentResults.Classifications)
+                {
+                    Console.WriteLine($"  Class category \"{classification.Category}\" predicted with a confidence score of {classification.ConfidenceScore}.");
+                }
+
+                Console.WriteLine();
+            }
+        }
+    }
+}
+```
+
+To see the full example source files, see:
+
+* [Synchronously Multi Category Classify](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample11_MultiCategoryClassify.cs)
+* [Asynchronously Multi Category Classify](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample11_MultiCategoryClassifyAsync.cs)
+* [Synchronously Multi Category Classify Convenience](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample11_MultiCategoryClassifyConvenience.cs)
+* [Asynchronously Multi Category Classify Convenience](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample11_MultiCategoryClassifyConvenienceAsync.cs)
+
+[DefaultAzureCredential]: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/identity/Azure.Identity/README.md
+[README]: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/README.md
diff --git a/...textanalytics/Azure.AI.TextAnalytics/samples/Sample9_RecognizeCustomEntities.md b/...textanalytics/Azure.AI.TextAnalytics/samples/Sample9_RecognizeCustomEntities.md
@@ -0,0 +1,113 @@
+# Recognizing Custom Entities from Documents
+This sample demonstrates how to recognize custom entities in one or more documents. To get started you'll need a Text Analytics endpoint and credentials.  See [README][README] for links and instructions.
+
+## Creating a `TextAnalyticsClient`
+
+To create a new `TextAnalyticsClient` to recognize custom entities in a document, you need a Text Analytics endpoint and credentials.  You can use the [DefaultAzureCredential][DefaultAzureCredential] to try a number of common authentication methods optimized for both running as a service and development.  In the sample below, however, you'll use a Text Analytics API key credential by creating an `AzureKeyCredential` object, that if needed, will allow you to update the API key without creating a new client.
+
+You can set `endpoint` and `apiKey` based on an environment variable, a configuration setting, or any way that works for your application.
+
+```C# Snippet:CreateTextAnalyticsClient
+string endpoint = "<endpoint>";
+string apiKey = "<apiKey>";
+var client = new TextAnalyticsClient(new Uri(endpoint), new AzureKeyCredential(apiKey));
+```
+
+## Recognizing custom entities in a single or multiple documents
+
+To recognize custom entities in documents, set up a `RecognizeCustomEntitiesAction` and call `StartAnalyzeActionsAsync` on the documents. The result is a Long Running operation of type `AnalyzeActionsOperation` which polls for the results from the API. You can use [Azure language studio][azure_language_studio] to train custom models.
+
+```C# Snippet:RecognizeCustomEntitiesActionAsync
+// Create input documents.
+string documentA = @"We love this trail and make the trip every year. The views are breathtaking and well
+                    worth the hike! Yesterday was foggy though, so we missed the spectacular views.
+                    We tried again today and it was amazing. Everyone in my family liked the trail although
+                    it was too challenging for the less athletic among us.";
+
+string documentB = @"Last week we stayed at Hotel Foo to celebrate our anniversary. The staff knew about
+                    our anniversary so they helped me organize a little surprise for my partner.
+                    The room was clean and with the decoration I requested. It was perfect!";
+
+var batchDocuments = new List<TextDocumentInput>
+{
+    new TextDocumentInput("1", documentA)
+    {
+         Language = "en",
+    },
+    new TextDocumentInput("2", documentB)
+    {
+         Language = "en",
+    }
+};
+
+// prepare actions.
+string projectName = "<projectName>";
+string deploymentName = "<deploymentName>";
+var actions = new TextAnalyticsActions()
+{
+    RecognizeCustomEntitiesActions = new List<RecognizeCustomEntitiesAction>()
+    {
+        new RecognizeCustomEntitiesAction(projectName, deploymentName);
+    }
+};
+
+AnalyzeActionsOperation operation = await client.StartAnalyzeActionsAsync(batchDocuments, actions);
+
+await operation.WaitForCompletionAsync();
+```
+
+The returned `AnalyzeActionsOperation` contains general information about the status of the operation. It can be requested while the operation is running or when it has completed. For example:
+
+```C# Snippet:RecognizeCustomEntitiesActionOperationStatus
+// View operation status.
+Console.WriteLine($"AnalyzeActions operation has completed");
+Console.WriteLine();
+Console.WriteLine($"Created On   : {operation.CreatedOn}");
+Console.WriteLine($"Expires On   : {operation.ExpiresOn}");
+Console.WriteLine($"Id           : {operation.Id}");
+Console.WriteLine($"Status       : {operation.Status}");
+Console.WriteLine($"Last Modified: {operation.LastModified}");
+Console.WriteLine();
+```
+
+To view the final results of the long-running operation:
+
+```C# Snippet:RecognizeCustomEntitiesActionAsyncViewResults
+await foreach (AnalyzeActionsResult documentsInPage in operation.Value)
+{
+    IReadOnlyCollection<RecognizeCustomEntitiesActionResult> customEntitiesActionResults = documentsInPage.RecognizeCustomEntitiesResults;
+    foreach (RecognizeCustomEntitiesActionResult customEntitiesActionResult in customEntitiesActionResults)
+    {
+        Console.WriteLine($" Action name: {customEntitiesActionResult.ActionName}");
+        int docNumber = 1;
+        foreach (RecognizeEntitiesResult documentResults in customEntitiesActionResult.DocumentsResults)
+        {
+            Console.WriteLine($" Document #{docNumber++}");
+            Console.WriteLine($"  Recognized the following {documentResults.Entities.Count} entities:");
+
+            foreach (CategorizedEntity entity in documentResults.Entities)
+            {
+                Console.WriteLine($"  Entity: {entity.Text}");
+                Console.WriteLine($"  Category: {entity.Category}");
+                Console.WriteLine($"  Offset: {entity.Offset}");
+                Console.WriteLine($"  Length: {entity.Length}");
+                Console.WriteLine($"  ConfidenceScore: {entity.ConfidenceScore}");
+                Console.WriteLine($"  SubCategory: {entity.SubCategory}");
+            }
+            Console.WriteLine("");
+        }
+    }
+}
+```
+
+To see the full example source files, see:
+
+* [Synchronously RecognizeCustomEntities](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample9_RecognizeCustomEntities.cs)
+* [Asynchronously RecognizeCustomEntities](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples//Sample9_RecognizeCustomEntitiesAsync.cs)
+* [Synchronously RecognizeCustomEntities Convenience](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample9_RecognizeCustomEntitiesConvenience.cs)
+* [Asynchronously RecognizeCustomEntities Convenience](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample9_RecognizeCustomEntitiesConvenienceAsync.cs)
+
+<!-- LINKS -->
+[azure_language_studio]: https://language.azure.com/
+[README]: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/textanalytics/Azure.AI.TextAnalytics/README.md
+[DefaultAzureCredential]: https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/identity/Azure.Identity/README.md
diff --git a/sdk/textanalytics/Azure.AI.TextAnalytics/samples/Sample_AnalyzeActions.md b/sdk/textanalytics/Azure.AI.TextAnalytics/samples/Sample_AnalyzeActions.md
@@ -64,6 +64,7 @@ To run multiple actions in multiple documents, call `StartAnalyzeActionsAsync` o
         int docNumber = 1;
         foreach (RecognizeEntitiesActionResult entitiesActionResults in entitiesResults)
         {
+            Console.WriteLine($" Action name: {entitiesActionResults.ActionName}");
             foreach (RecognizeEntitiesResult documentResults in entitiesActionResults.DocumentsResults)
             {
                 Console.WriteLine($" Document #{docNumber++}");