Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Cloud Natural Language API Java sample. #275

Merged
merged 1 commit into from
Jul 20, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions language/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Google Cloud Natural Language API Samples

These samples demonstrate the use of the [Google Cloud Natural Language API][NL-Docs].

[NL-Docs]: https://cloud.google.com/language/docs/

## Prerequisites

### Download Maven

This sample uses the [Apache Maven][maven] build system. Before getting started, be
sure to [download][maven-download] and [install][maven-install] it. When you use
Maven as described here, it will automatically download the needed client
libraries.

[maven]: https://maven.apache.org
[maven-download]: https://maven.apache.org/download.cgi
[maven-install]: https://maven.apache.org/install.html

### Set Up to Authenticate With Your Project's Credentials

Please follow the [Set Up Your Project](https://cloud.google.com/natural-language/docs/getting-started#set_up_your_project)
steps in the Quickstart doc to create a project and enable the
Cloud Natural Language API. Following those steps, make sure that you
[Set Up a Service Account](https://cloud.google.com/natural-language/docs/common/auth#set_up_a_service_account),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This really isn't necessary in most instances. But I won't block on it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dunno - I feel like it's the most failsafe way to do things. The other methods of getting credentials have too many caveats attached to them :-/

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a huge fan of gcloud for credentials and DefaultAuth.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here my (possibly outdated) understanding:

gcloud

  • auth login which works sometimes but fails subtly because it uses user auth on the cloudsdktool project instead of a service account that's attached to the project. Also you'll have to swap this out with a service account anyway once you go to production, if you're not on GCE/GAE.
  • auth activate-service-account which requires downloading a service account anyway, and what happens in your production scripts if you're not on GCE/GAE? I suppose you could have your machine run the gcloud command as part of setup? Seems like more overhead than setting an env variable..
  • In general, requires installing gcloud on every machine you run the code on (unless you're on GAE/GCE)

I assume by DefaultAuth you mean the default service account on GAE/GCE? Which is fine, unless you're not on GAE/GCE, in which case we have to mention that and explain what to do in that case, so it doesn't save any explanation space (though yes - if you are running on GAE/GCE, you can just skip that paragraph).

Point being - it's fine, but there's a lot of explaining and decision points around whether you're on GAE/GCE, that folks probably don't care about when they're looking at a sample like this, so it's simpler to just have one not-too-complex explanation that works in all instances :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gcloud beta auth application-default is the latest hotness.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image of Yaktocat

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately doesn't seem to work :(

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should let Vijay know.

and export the following environment variable:

```
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/your-project-credentials.json
```

[cloud-console]: https://console.cloud.google.com
[language-api]: https://console.cloud.google.com/apis/api/language.googleapis.com/overview?project=_
[adc]: https://cloud.google.com/docs/authentication#developer_workflow

## Samples

- [Analyze](analysis) is a command line tool to show case the features of the API.

53 changes: 53 additions & 0 deletions language/analysis/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Google Cloud Natural Language API Entity Recognition Sample

This sample demonstrates the use of the [Google Cloud Natural Language API][NL-Docs]
for entity recognition.

[NL-Docs]: https://cloud.google.com/language/docs/

## Java Version

This sample requires you to have
[Java8](https://docs.oracle.com/javase/8/docs/technotes/guides/install/install_overview.html).

## Download Maven

This sample uses the [Apache Maven][maven] build system. Before getting started, be
sure to [download][maven-download] and [install][maven-install] it. When you use
Maven as described here, it will automatically download the needed client
libraries.

[maven]: https://maven.apache.org
[maven-download]: https://maven.apache.org/download.cgi
[maven-install]: https://maven.apache.org/install.html

## Run the sample

To build the sample, we use Maven.

```bash
mvn clean compile assembly:single
```

We can then run the assembled JAR file with the `java` command. The variable $COMMAND takes
three values `entities`, `sentiment` or `syntax`.

```
MAIN_CLASS=com.google.cloud.language.samples.Analyze
JAR_FILE=target/entities-1.0-SNAPSHOT-jar-with-dependencies.jar
java -cp $JAR_FILE $MAIN_CLASS <sentiment|entities|syntax> <text>
```

Example usage:

```
QUOTE="Larry Page, Google's co-founder, once described the 'perfect search
engine' as something that 'understands exactly what you mean and gives you
back exactly what you want.' Since he spoke those words Google has grown to
offer products beyond search, but the spirit of what he said remains."

java -cp $JAR_FILE $MAIN_CLASS entities "$QUOTE"
java -cp $JAR_FILE $MAIN_CLASS sentiment "$QUOTE"
java -cp $JAR_FILE $MAIN_CLASS syntax "$QUOTE"
```

93 changes: 93 additions & 0 deletions language/analysis/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
<!--
Copyright 2016 Google Inc. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<project>
<modelVersion>4.0.0</modelVersion>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<groupId>com.google.cloud.language.samples</groupId>
<artifactId>entities</artifactId>

<dependencies>
<!-- [START dependencies] -->
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-language</artifactId>
<version>v1beta1-rev1-1.22.0</version>
</dependency>
<dependency>
<groupId>com.google.api-client</groupId>
<artifactId>google-api-client</artifactId>
<version>1.21.0</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>19.0</version>
</dependency>
<!-- [END dependencies] -->

<!-- Test Dependencies -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
<dependency>
<groupId>com.google.truth</groupId>
<artifactId>truth</artifactId>
<version>0.28</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>com.google.cloud.language.samples.entities.AnalyzeEntitiesApp</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<version>3.3</version>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,209 @@
/*
* Copyright 2016 Google Inc. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.google.cloud.language.samples;

import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.http.HttpRequest;
import com.google.api.client.http.HttpRequestInitializer;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.services.language.v1beta1.CloudNaturalLanguageAPI;
import com.google.api.services.language.v1beta1.CloudNaturalLanguageAPIScopes;
import com.google.api.services.language.v1beta1.model.AnalyzeEntitiesRequest;
import com.google.api.services.language.v1beta1.model.AnalyzeEntitiesResponse;
import com.google.api.services.language.v1beta1.model.AnalyzeSentimentRequest;
import com.google.api.services.language.v1beta1.model.AnalyzeSentimentResponse;
import com.google.api.services.language.v1beta1.model.AnnotateTextRequest;
import com.google.api.services.language.v1beta1.model.AnnotateTextResponse;
import com.google.api.services.language.v1beta1.model.Document;
import com.google.api.services.language.v1beta1.model.Entity;
import com.google.api.services.language.v1beta1.model.Features;
import com.google.api.services.language.v1beta1.model.Sentiment;
import com.google.api.services.language.v1beta1.model.Token;

import java.io.IOException;
import java.io.PrintStream;
import java.security.GeneralSecurityException;
import java.util.List;
import java.util.Map;

/**
* A sample application that uses the Natural Language API to perform
* entity, sentiment and syntax analysis.
*/
@SuppressWarnings("serial")
public class Analyze {
/**
* Be sure to specify the name of your application. If the application name is {@code null} or
* blank, the application will log a warning. Suggested format is "MyCompany-ProductName/1.0".
*/
private static final String APPLICATION_NAME = "Google-LanguagAPISample/1.0";

private static final int MAX_RESULTS = 4;

/**
* Detects entities,sentiment and syntax in a document using the Natural Language API.
*/
public static void main(String[] args) throws IOException, GeneralSecurityException {
if (args.length != 2) {
System.err.println("Usage:");
System.err.printf(
"\tjava %s \"command\" \"text to analyze\"\n",
Analyze.class.getCanonicalName());
System.exit(1);
}
String command = args[0];
String text = args[1];

Analyze app = new Analyze(getLanguageService());

if (command.equals("entities")) {
printEntities(System.out, app.analyzeEntities(text));
} else if (command.equals("sentiment")) {
printSentiment(System.out, app.analyzeSentiment(text));
} else if (command.equals("syntax")) {
printSyntax(System.out, app.analyzeSyntax(text));
}
}

/**
* Print a list of {@code entities}.
*/
public static void printEntities(PrintStream out, List<Entity> entities) {
if (entities == null || entities.size() == 0) {
out.println("No entities found.");
return;
}
out.printf("Found %d entit%s.\n", entities.size(), entities.size() == 1 ? "y" : "ies");
for (Entity entity : entities) {
out.printf("%s\n", entity.getName());
out.printf("\tSalience: %.3f\n", entity.getSalience());
out.printf("\tType: %s\n", entity.getType());
if (entity.getMetadata() != null) {
for (Map.Entry<String, String> metadata : entity.getMetadata().entrySet()) {
out.printf("\tMetadata: %s = %s\n", metadata.getKey(), metadata.getValue());
}
}
}
}

/**
* Print the Sentiment {@code sentiment}.
*/
public static void printSentiment(PrintStream out, Sentiment sentiment) {
if (sentiment == null) {
out.println("No sentiment found");
return;
}
out.println("Found sentiment.");
out.printf("\tMagnitude: %.3f\n", sentiment.getMagnitude());
out.printf("\tPolarity: %.3f\n", sentiment.getPolarity());
}

public static void printSyntax(PrintStream out, List<Token> tokens) {
if (tokens == null || tokens.size() == 0) {
out.println("No syntax found");
return;
}
out.printf("Found %d token%s.\n", tokens.size(), tokens.size() == 1 ? "" : "s");
for (Token token : tokens) {
out.println("TextSpan");
out.printf("\tText: %s\n", token.getText().getContent());
out.printf("\tBeginOffset: %d\n", token.getText().getBeginOffset());
out.printf("Lemma: %s\n", token.getLemma());
out.printf("PartOfSpeechTag: %s\n", token.getPartOfSpeech().getTag());
out.println("DependencyEdge");
out.printf("\tHeadTokenIndex: %d\n", token.getDependencyEdge().getHeadTokenIndex());
out.printf("\tLabel: %s\n", token.getDependencyEdge().getLabel());
}
}

/**
* Connects to the Natural Language API using Application Default Credentials.
*/
public static CloudNaturalLanguageAPI getLanguageService()
throws IOException, GeneralSecurityException {
GoogleCredential credential =
GoogleCredential.getApplicationDefault().createScoped(CloudNaturalLanguageAPIScopes.all());
JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
return new CloudNaturalLanguageAPI.Builder(
GoogleNetHttpTransport.newTrustedTransport(),
jsonFactory, new HttpRequestInitializer() {
@Override
public void initialize(HttpRequest request) throws IOException {
credential.initialize(request);
}
})
.setApplicationName(APPLICATION_NAME)
.build();
}

private final CloudNaturalLanguageAPI languageApi;

/**
* Constructs a {@link Analyze} which connects to the Cloud Natural Language API.
*/
public Analyze(CloudNaturalLanguageAPI languageApi) {
this.languageApi = languageApi;
}

/**
* Gets {@link Entity}s from the string {@code text}.
*/
public List<Entity> analyzeEntities(String text) throws IOException {
AnalyzeEntitiesRequest request =
new AnalyzeEntitiesRequest()
.setDocument(new Document().setContent(text).setType("PLAIN_TEXT"))
.setEncodingType("UTF16");
CloudNaturalLanguageAPI.Documents.AnalyzeEntities analyze =
languageApi.documents().analyzeEntities(request);

AnalyzeEntitiesResponse response = analyze.execute();
return response.getEntities();
}

/**
* Gets {@link Sentiment} from the string {@code text}.
*/
public Sentiment analyzeSentiment(String text) throws IOException {
AnalyzeSentimentRequest request =
new AnalyzeSentimentRequest()
.setDocument(new Document().setContent(text).setType("PLAIN_TEXT"));
CloudNaturalLanguageAPI.Documents.AnalyzeSentiment analyze =
languageApi.documents().analyzeSentiment(request);

AnalyzeSentimentResponse response = analyze.execute();
return response.getDocumentSentiment();
}

/**
* Gets {@link Token}s from the string {@code text}.
*/
public List<Token> analyzeSyntax(String text) throws IOException {
AnnotateTextRequest request =
new AnnotateTextRequest()
.setDocument(new Document().setContent(text).setType("PLAIN_TEXT"))
.setFeatures(new Features().setExtractSyntax(true))
.setEncodingType("UTF16");
CloudNaturalLanguageAPI.Documents.AnnotateText analyze =
languageApi.documents().annotateText(request);

AnnotateTextResponse response = analyze.execute();
return response.getTokens();
}
}
Loading