Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Language Text] Add support for v2022-10-01-preview #23208

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion sdk/cognitivelanguage/ai-language-text/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,18 @@
# Release History

## 1.0.1 (Unreleased)
## 1.1.0-beta.1 (Unreleased)

### Features Added

- Supports service version 2022-10-01-preview by default instead of 2022-05-01.
- Adds support for extractive summarization and FHIR.
- Adds support for abstractive summarization.
- Adds support for dynamic classification.
- Adds support for script detection and automatic language detection.
- Adds support for document types in healthcare analysis.
- Adds support for entity resolution in entity recognition.
- Adds support for confidence scores in healthcare relations.

### Breaking Changes

### Bugs Fixed
Expand Down
145 changes: 128 additions & 17 deletions sdk/cognitivelanguage/ai-language-text/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[Azure Cognitive Service for Language](https://azure.microsoft.com/services/cognitive-services/language-service/) is a cloud-based service that provides advanced natural language processing over raw text, and includes the following main features:

**Note:** This SDK targets Azure Cognitive Service for Language API version 2022-05-01.
**Note:** This SDK targets Azure Cognitive Service for Language API version 2022-10-01-preview.

- Language Detection
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given how long this list is, if we're going to keep putting every feature our SDK supports in the README, we should probably start making these links to bookmarks. GitHub does generate a TOC, but it's hidden and, in my experience, not a lot of people know about it. Besides, some of these READMEs get rendered in package repo sites across different languages. Bookmarks should still work there.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By "links to bookmarks" do you mean linked to the service docs pages?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, e.g.,

- [link text](#link-bookmark)

## Link Bookmark

Headings are automatically transformed into slugs, though with words like "a", "an", "the", etc., some renderers remove them. If that's a concern, be explicit:

## A Link Bookmark<a name="link-bookmark"></a>

My worry is that with so many features - and always more - with code snippets, the README is getting so large it's harder to navigate and quite overwhelming without either breaking it up, or at least making it easier to jump to interesting sections for the reader.

Copy link
Member

@kristapratico kristapratico Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah agreed, we got this feedback awhile back: Azure/azure-sdk-for-python#19435 (from Johan IIRC). In Python, we haven't been adding a Readme example with every new feature, instead we link out specifically to the sample. edit -- under the Examples section, though, not under the first header

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good points, I guess the main question here is how to present all the features on an equal footing and still be short and sweet. How does everyone feel about Python's?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened #23242 to track this.

- Sentiment Analysis
Expand All @@ -11,8 +11,11 @@
- Recognition of Personally Identifiable Information
- Entity Linking
- Healthcare Analysis
- Extractive Summarization
- Abstractive Summarization
- Custom Entity Recognition
- Custom Document Classification
- Dynamic Classification
- Support Multiple Actions Per Document

Use the client library to:
Expand Down Expand Up @@ -389,15 +392,48 @@ async function main() {
main();
```

### Dynamic Classification

On the fly classification of the input documents into one or multiple categories. Assigns either one or multiple categories per document. This type of classification doesn't require model training.
heaths marked this conversation as resolved.
Show resolved Hide resolved

```javascript
const { TextAnalysisClient, AzureKeyCredential } = require("@azure/ai-language-text");

const client = new TextAnalysisClient("<endpoint>", new AzureKeyCredential("<API key>"));

const documents = [
"The WHO is issuing a warning about Monkey Pox.",
"Mo Salah plays in Liverpool FC in England.",
];

export async function main() {
const results = await client.analyze("DynamicClassification", documents, "en", {
categories: ["Health", "Politics", "Music", "Sports"],
classificationType: "Multi",
});

for (const result of results) {
console.log(`- Document ${result.id}`);
if (!result.error) {
console.log("\tClassifications:");
for (const category of result.classifications) {
console.log(`\t- ${category.category}`);
}
} else {
console.error(" Error:", result.error);
}
}
}

main();
```

### Healthcare Analysis

Healthcare analysis identifies healthcare entities. For example, given input text "Prescribed 100mg ibuprofen, taken twice daily", the service returns "100mg" categorized as Dosage, "ibuprofen" as MedicationName, and "twice daily" as Frequency.

```javascript
const {
AzureKeyCredential,
TextAnalysisClient,
} = require("@azure/ai-language-text");
const { AzureKeyCredential, TextAnalysisClient } = require("@azure/ai-language-text");

const client = new TextAnalysisClient("<endpoint>", new AzureKeyCredential("<API key>"));

Expand Down Expand Up @@ -445,15 +481,96 @@ async function main() {
main();
```

### Extractive Summarization

Extractive summarization identifies sentences that summarize the article they belong to.

```javascript
const { AzureKeyCredential, TextAnalysisClient } = require("@azure/ai-language-text");
const client = new TextAnalysisClient("<endpoint>", new AzureKeyCredential("<API key>"));
const documents = [
"<long article>"
];
async function main() {
const actions = [
{
kind: "ExtractiveSummarization",
maxSentenceCount: 2,
},
];
const poller = await client.beginAnalyzeBatch(actions, documents, "en");
const results = await poller.pollUntilDone();
for await (const actionResult of results) {
if (actionResult.kind !== "ExtractiveSummarization") {
throw new Error(`Expected extractive summarization results but got: ${actionResult.kind}`);
}
if (actionResult.error) {
const { code, message } = actionResult.error;
throw new Error(`Unexpected error (${code}): ${message}`);
}
for (const result of actionResult.results) {
console.log(`- Document ${result.id}`);
if (result.error) {
const { code, message } = result.error;
throw new Error(`Unexpected error (${code}): ${message}`);
}
console.log("Summary:");
console.log(result.sentences.map((sentence) => sentence.text).join("\n"));
}
}
}
main();
```

### Abstractive Summarization

Abstractive summarization generates a summary for the input articles. Abstractive summarization is different from extractive summarization in that extractive summarization is the strategy of concatenating extracted sentences from the input document into a summary, while abstractive summarization involves paraphrasing the document using novel sentences.

```javascript
const { AzureKeyCredential, TextAnalysisClient } = require("@azure/ai-language-text");
const client = new TextAnalysisClient("<endpoint>", new AzureKeyCredential("<API key>"));
const documents = [
"<long article>"
];
async function main() {
const actions = [
{
kind: "AbstractiveSummarization",
maxSentenceCount: 2,
},
];
const poller = await client.beginAnalyzeBatch(actions, documents, "en");
const results = await poller.pollUntilDone();
for await (const actionResult of results) {
if (actionResult.kind !== "AbstractiveSummarization") {
throw new Error(`Expected abstractive summarization results but got: ${actionResult.kind}`);
}
if (actionResult.error) {
const { code, message } = actionResult.error;
throw new Error(`Unexpected error (${code}): ${message}`);
}
for (const result of actionResult.results) {
console.log(`- Document ${result.id}`);
if (result.error) {
const { code, message } = result.error;
throw new Error(`Unexpected error (${code}): ${message}`);
}
console.log("\t- Summary:");
for (const summary of result.summaries) {
console.log(summary.text);
}
}
}
}
main();
```

### Custom Entity Recognition

Recognize and categorize entities in text as entities using custom entity detection models built using [Azure Language Studio][lang_studio].

```javascript
const {
AzureKeyCredential,
TextAnalysisClient,
} = require("@azure/ai-language-text");
const { AzureKeyCredential, TextAnalysisClient } = require("@azure/ai-language-text");

const client = new TextAnalysisClient("<endpoint>", new AzureKeyCredential("<API key>"));

Expand Down Expand Up @@ -549,10 +666,7 @@ main();
Classify documents using custom multi-label models built using [Azure Language Studio][lang_studio].

```javascript
const {
AzureKeyCredential,
TextAnalysisClient,
} = require("@azure/ai-language-text");
const { AzureKeyCredential, TextAnalysisClient } = require("@azure/ai-language-text");

const client = new TextAnalysisClient("<endpoint>", new AzureKeyCredential("<API key>"));

Expand Down Expand Up @@ -603,10 +717,7 @@ main();
Applies multiple actions on each input document in one service request.

```javascript
const {
AzureKeyCredential,
TextAnalysisClient,
} = require("@azure/ai-language-text");
const { AzureKeyCredential, TextAnalysisClient } = require("@azure/ai-language-text");

const client = new TextAnalysisClient("<endpoint>", new AzureKeyCredential("<API key>"));

Expand Down
2 changes: 1 addition & 1 deletion sdk/cognitivelanguage/ai-language-text/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"sdk-type": "client",
"author": "Microsoft Corporation",
"description": "An isomorphic client library for the text analysis features in the Azure Cognitive Language Service.",
"version": "1.0.1",
"version": "1.1.0-beta.1",
"keywords": [
"node",
"azure",
Expand Down
Loading