-
Notifications
You must be signed in to change notification settings - Fork 4.9k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[TA] Samples and documentation for Opinion Mining and Pii (#14829)
* samples, readme + fix version * update links * update samples
- Loading branch information
Showing
13 changed files
with
561 additions
and
13 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
93 changes: 93 additions & 0 deletions
93
...s/Azure.AI.TextAnalytics/samples/Sample2.1_AnalyzeSentimentWithOpinionMining.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| # Analyze sentiment with Opinion Mining | ||
|
|
||
| This sample demonstrates how to analyze sentiment of documents and get more granular information about the opinions related to aspects of a product/service, also knows as Aspect-based Sentiment Analysis in Natural Language Processing (NLP). This feature is only available for clients with api version v3.1-preview.1 and higher. | ||
|
|
||
| For the purpose of the sample, we will be the administrator of a hotel and we've set a system to look at the online reviews customers are posting to identify the major complaints about our hotel. | ||
| In order to do so, we will use the Sentiment Analysis feature of the Text Analytics client library. To get started you'll need a Text Analytics endpoint and credentials. See [README][README] for links and instructions. | ||
|
|
||
| ## Creating a `TextAnalyticsClient` | ||
|
|
||
| To create a new `TextAnalyticsClient`, you need a Text Analytics endpoint and credentials. You can use the [DefaultAzureCredential][DefaultAzureCredential] to try a number of common authentication methods optimized for both running as a service and development. In the sample below, however, you'll use a Text Analytics API key credential by creating an `AzureKeyCredential` object, that if needed, will allow you to update the API key without creating a new client. | ||
|
|
||
| You can set `endpoint` and `apiKey` based on an environment variable, a configuration setting, or any way that works for your application. | ||
|
|
||
| ```C# Snippet:TextAnalyticsSample1CreateClient | ||
| var client = new TextAnalyticsClient(new Uri(endpoint), new AzureKeyCredential(apiKey)); | ||
| ``` | ||
|
|
||
| ## Identify complaints | ||
|
|
||
| To get a deeper analysis into which are the aspects that people considered good or bad, we will need to include the `AdditionalSentimentAnalyses.OpinionMining` type into the `AnalyzeSentimentOptions`. | ||
|
|
||
| ```C# Snippet:TAAnalyzeSentimentWithOpinionMining | ||
| var documents = new List<string> | ||
| { | ||
| "The food and service were unacceptable, but the concierge were nice.", | ||
| "The rooms were beautiful. The AC was good and quiet.", | ||
| "The breakfast was good, but the toilet was smelly.", | ||
| "Loved this hotel - good breakfast - nice shuttle service - clean rooms.", | ||
| "I had a great unobstructed view of the Microsoft campus.", | ||
| "Nice rooms but bathrooms were old and the toilet was dirty when we arrived.", | ||
| "We changed rooms as the toilet smelled." | ||
| }; | ||
|
|
||
| AnalyzeSentimentResultCollection reviews = client.AnalyzeSentimentBatch(documents, options: new AnalyzeSentimentOptions() { AdditionalSentimentAnalyses = AdditionalSentimentAnalyses.OpinionMining }); | ||
|
|
||
| Dictionary<string, int> complaints = GetComplaints(reviews); | ||
|
|
||
| var negativeAspect = complaints.Aggregate((l, r) => l.Value > r.Value ? l : r).Key; | ||
| Console.WriteLine($"Alert! major complaint is *{negativeAspect}*"); | ||
| Console.WriteLine(); | ||
| Console.WriteLine("---All complaints:"); | ||
| foreach (KeyValuePair<string, int> complaint in complaints) | ||
| { | ||
| Console.WriteLine($" {complaint.Key}, {complaint.Value}"); | ||
| } | ||
| ``` | ||
|
|
||
| Output: | ||
| ``` | ||
| Alert! major complaint is *toilet* | ||
| ---All complaints: | ||
| food, 1 | ||
| service, 1 | ||
| toilet, 3 | ||
| bathrooms, 1 | ||
| rooms, 1 | ||
| ``` | ||
|
|
||
| ## Define method `GetComplaints` | ||
| Implementation for calculating complaints: | ||
|
|
||
| ```C# Snippet:TAGetComplaints | ||
| private Dictionary<string, int> GetComplaints(AnalyzeSentimentResultCollection reviews) | ||
| { | ||
| var complaints = new Dictionary<string, int>(); | ||
| foreach (AnalyzeSentimentResult review in reviews) | ||
| { | ||
| foreach (SentenceSentiment sentence in review.DocumentSentiment.Sentences) | ||
| { | ||
| foreach (MinedOpinion minedOpinion in sentence.MinedOpinions) | ||
| { | ||
| if (minedOpinion.Aspect.Sentiment == TextSentiment.Negative) | ||
| { | ||
| complaints.TryGetValue(minedOpinion.Aspect.Text, out var value); | ||
| complaints[minedOpinion.Aspect.Text] = value + 1; | ||
| } | ||
| } | ||
| } | ||
| } | ||
| return complaints; | ||
| } | ||
| ``` | ||
|
|
||
|
|
||
| To see the full example source files, see: | ||
| * [Synchronous Analyze Sentiment with Opinion Mining](https://github.com/maririos/azure-sdk-for-net/blob/cd74dde9546ea675f9289e1b4e6fd804bda2a3bc/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample2.1_AnalyzeSentimentWithOpinionMining.cs) | ||
| * [Asynchronous Analyze Sentiment with Opinion Mining](https://github.com/maririos/azure-sdk-for-net/blob/cd74dde9546ea675f9289e1b4e6fd804bda2a3bc/sdk/textanalytics/Azure.AI.TextAnalytics/tests/samples/Sample2.1_AnalyzeSentimentWithOpinionMiningAsync.cs) | ||
| <!--* [Synchronous Analyze Sentiment with Opinion Mining](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/textanalytics/Azure.AI.TextAnalytics//tests/samples/Sample2.1_AnalyzeSentimentWithOpinionMining.cs) | ||
| * [Asynchronous Analyze Sentiment with Opinion Mining](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/textanalytics/Azure.AI.TextAnalytics//tests/samples/Sample2.1_AnalyzeSentimentWithOpinionMiningAsync.cs)--> | ||
|
|
||
| [DefaultAzureCredential]: https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/identity/Azure.Identity/README.md | ||
| [README]: https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/textanalytics/Azure.AI.TextAnalytics/README.md |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
385d220There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have tested "Sample5_RecognizePiiEntities" with below string. Not sure why Credit Card Number, CVV and PIN are not getting redacted. Can you please help
string documentA = @"Good morning, everybody. My name is Van Bokhorst Serdar, and today I feel like sharing a whole lot of personal information with you.
Let's start with my Email address SerdarvanBokhorst@dayrep.com. My address is 2657 Koontz Lane, Los Angeles, CA. My phone number is 818-828-6231.
My Social security number is 548-95-6370. My Bank account number is 940517528812 and routing number 195991012. My credit card number is 5534816011668430,
Expiration Date 6/1/2022, my C V V code is 121, and my password 123456. Well, I think that's it. You know a whole lot about me. And I hope that Amazon comprehend is doing a
good job at identifying PII entities so you can redact my personal information away from this document. Let's check. IBN GB33BUKB20201555555555";