-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Cloud Natural Language API Java sample. #275
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
# Google Cloud Natural Language API Samples | ||
|
||
These samples demonstrate the use of the [Google Cloud Natural Language API][NL-Docs]. | ||
|
||
[NL-Docs]: https://cloud.google.com/language/docs/ | ||
|
||
## Prerequisites | ||
|
||
### Download Maven | ||
|
||
This sample uses the [Apache Maven][maven] build system. Before getting started, be | ||
sure to [download][maven-download] and [install][maven-install] it. When you use | ||
Maven as described here, it will automatically download the needed client | ||
libraries. | ||
|
||
[maven]: https://maven.apache.org | ||
[maven-download]: https://maven.apache.org/download.cgi | ||
[maven-install]: https://maven.apache.org/install.html | ||
|
||
### Set Up to Authenticate With Your Project's Credentials | ||
|
||
Please follow the [Set Up Your Project](https://cloud.google.com/natural-language/docs/getting-started#set_up_your_project) | ||
steps in the Quickstart doc to create a project and enable the | ||
Cloud Natural Language API. Following those steps, make sure that you | ||
[Set Up a Service Account](https://cloud.google.com/natural-language/docs/common/auth#set_up_a_service_account), | ||
and export the following environment variable: | ||
|
||
``` | ||
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/your-project-credentials.json | ||
``` | ||
|
||
[cloud-console]: https://console.cloud.google.com | ||
[language-api]: https://console.cloud.google.com/apis/api/language.googleapis.com/overview?project=_ | ||
[adc]: https://cloud.google.com/docs/authentication#developer_workflow | ||
|
||
## Samples | ||
|
||
- [Analyze](analysis) is a command line tool to show case the features of the API. | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,53 @@ | ||
# Google Cloud Natural Language API Entity Recognition Sample | ||
|
||
This sample demonstrates the use of the [Google Cloud Natural Language API][NL-Docs] | ||
for entity recognition. | ||
|
||
[NL-Docs]: https://cloud.google.com/language/docs/ | ||
|
||
## Java Version | ||
|
||
This sample requires you to have | ||
[Java8](https://docs.oracle.com/javase/8/docs/technotes/guides/install/install_overview.html). | ||
|
||
## Download Maven | ||
|
||
This sample uses the [Apache Maven][maven] build system. Before getting started, be | ||
sure to [download][maven-download] and [install][maven-install] it. When you use | ||
Maven as described here, it will automatically download the needed client | ||
libraries. | ||
|
||
[maven]: https://maven.apache.org | ||
[maven-download]: https://maven.apache.org/download.cgi | ||
[maven-install]: https://maven.apache.org/install.html | ||
|
||
## Run the sample | ||
|
||
To build the sample, we use Maven. | ||
|
||
```bash | ||
mvn clean compile assembly:single | ||
``` | ||
|
||
We can then run the assembled JAR file with the `java` command. The variable $COMMAND takes | ||
three values `entities`, `sentiment` or `syntax`. | ||
|
||
``` | ||
MAIN_CLASS=com.google.cloud.language.samples.Analyze | ||
JAR_FILE=target/entities-1.0-SNAPSHOT-jar-with-dependencies.jar | ||
java -cp $JAR_FILE $MAIN_CLASS <sentiment|entities|syntax> <text> | ||
``` | ||
|
||
Example usage: | ||
|
||
``` | ||
QUOTE="Larry Page, Google's co-founder, once described the 'perfect search | ||
engine' as something that 'understands exactly what you mean and gives you | ||
back exactly what you want.' Since he spoke those words Google has grown to | ||
offer products beyond search, but the spirit of what he said remains." | ||
|
||
java -cp $JAR_FILE $MAIN_CLASS entities "$QUOTE" | ||
java -cp $JAR_FILE $MAIN_CLASS sentiment "$QUOTE" | ||
java -cp $JAR_FILE $MAIN_CLASS syntax "$QUOTE" | ||
``` | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,93 @@ | ||
<!-- | ||
Copyright 2016 Google Inc. All Rights Reserved. | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
--> | ||
<project> | ||
<modelVersion>4.0.0</modelVersion> | ||
<packaging>jar</packaging> | ||
<version>1.0-SNAPSHOT</version> | ||
<groupId>com.google.cloud.language.samples</groupId> | ||
<artifactId>entities</artifactId> | ||
|
||
<dependencies> | ||
<!-- [START dependencies] --> | ||
<dependency> | ||
<groupId>com.google.apis</groupId> | ||
<artifactId>google-api-services-language</artifactId> | ||
<version>v1beta1-rev1-1.22.0</version> | ||
</dependency> | ||
<dependency> | ||
<groupId>com.google.api-client</groupId> | ||
<artifactId>google-api-client</artifactId> | ||
<version>1.21.0</version> | ||
</dependency> | ||
<dependency> | ||
<groupId>com.google.guava</groupId> | ||
<artifactId>guava</artifactId> | ||
<version>19.0</version> | ||
</dependency> | ||
<!-- [END dependencies] --> | ||
|
||
<!-- Test Dependencies --> | ||
<dependency> | ||
<groupId>junit</groupId> | ||
<artifactId>junit</artifactId> | ||
<version>4.12</version> | ||
</dependency> | ||
<dependency> | ||
<groupId>com.google.truth</groupId> | ||
<artifactId>truth</artifactId> | ||
<version>0.28</version> | ||
</dependency> | ||
</dependencies> | ||
<build> | ||
<plugins> | ||
<plugin> | ||
<artifactId>maven-assembly-plugin</artifactId> | ||
<configuration> | ||
<archive> | ||
<manifest> | ||
<mainClass>com.google.cloud.language.samples.entities.AnalyzeEntitiesApp</mainClass> | ||
</manifest> | ||
</archive> | ||
<descriptorRefs> | ||
<descriptorRef>jar-with-dependencies</descriptorRef> | ||
</descriptorRefs> | ||
</configuration> | ||
</plugin> | ||
<plugin> | ||
<groupId>org.apache.maven.plugins</groupId> | ||
<artifactId>maven-failsafe-plugin</artifactId> | ||
<version>2.18.1</version> | ||
<executions> | ||
<execution> | ||
<goals> | ||
<goal>integration-test</goal> | ||
<goal>verify</goal> | ||
</goals> | ||
</execution> | ||
</executions> | ||
</plugin> | ||
<plugin> | ||
<groupId>org.apache.maven.plugins</groupId> | ||
<version>3.3</version> | ||
<artifactId>maven-compiler-plugin</artifactId> | ||
<configuration> | ||
<source>1.8</source> | ||
<target>1.8</target> | ||
</configuration> | ||
</plugin> | ||
</plugins> | ||
</build> | ||
</project> |
209 changes: 209 additions & 0 deletions
209
language/analysis/src/main/java/com/google/cloud/language/samples/Analyze.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,209 @@ | ||
/* | ||
* Copyright 2016 Google Inc. All Rights Reserved. | ||
* | ||
* Licensed under the Apache License, Version 2.0 (the "License"); | ||
* you may not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*/ | ||
|
||
package com.google.cloud.language.samples; | ||
|
||
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential; | ||
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport; | ||
import com.google.api.client.http.HttpRequest; | ||
import com.google.api.client.http.HttpRequestInitializer; | ||
import com.google.api.client.json.JsonFactory; | ||
import com.google.api.client.json.jackson2.JacksonFactory; | ||
import com.google.api.services.language.v1beta1.CloudNaturalLanguageAPI; | ||
import com.google.api.services.language.v1beta1.CloudNaturalLanguageAPIScopes; | ||
import com.google.api.services.language.v1beta1.model.AnalyzeEntitiesRequest; | ||
import com.google.api.services.language.v1beta1.model.AnalyzeEntitiesResponse; | ||
import com.google.api.services.language.v1beta1.model.AnalyzeSentimentRequest; | ||
import com.google.api.services.language.v1beta1.model.AnalyzeSentimentResponse; | ||
import com.google.api.services.language.v1beta1.model.AnnotateTextRequest; | ||
import com.google.api.services.language.v1beta1.model.AnnotateTextResponse; | ||
import com.google.api.services.language.v1beta1.model.Document; | ||
import com.google.api.services.language.v1beta1.model.Entity; | ||
import com.google.api.services.language.v1beta1.model.Features; | ||
import com.google.api.services.language.v1beta1.model.Sentiment; | ||
import com.google.api.services.language.v1beta1.model.Token; | ||
|
||
import java.io.IOException; | ||
import java.io.PrintStream; | ||
import java.security.GeneralSecurityException; | ||
import java.util.List; | ||
import java.util.Map; | ||
|
||
/** | ||
* A sample application that uses the Natural Language API to perform | ||
* entity, sentiment and syntax analysis. | ||
*/ | ||
@SuppressWarnings("serial") | ||
public class Analyze { | ||
/** | ||
* Be sure to specify the name of your application. If the application name is {@code null} or | ||
* blank, the application will log a warning. Suggested format is "MyCompany-ProductName/1.0". | ||
*/ | ||
private static final String APPLICATION_NAME = "Google-LanguagAPISample/1.0"; | ||
|
||
private static final int MAX_RESULTS = 4; | ||
|
||
/** | ||
* Detects entities,sentiment and syntax in a document using the Natural Language API. | ||
*/ | ||
public static void main(String[] args) throws IOException, GeneralSecurityException { | ||
if (args.length != 2) { | ||
System.err.println("Usage:"); | ||
System.err.printf( | ||
"\tjava %s \"command\" \"text to analyze\"\n", | ||
Analyze.class.getCanonicalName()); | ||
System.exit(1); | ||
} | ||
String command = args[0]; | ||
String text = args[1]; | ||
|
||
Analyze app = new Analyze(getLanguageService()); | ||
|
||
if (command.equals("entities")) { | ||
printEntities(System.out, app.analyzeEntities(text)); | ||
} else if (command.equals("sentiment")) { | ||
printSentiment(System.out, app.analyzeSentiment(text)); | ||
} else if (command.equals("syntax")) { | ||
printSyntax(System.out, app.analyzeSyntax(text)); | ||
} | ||
} | ||
|
||
/** | ||
* Print a list of {@code entities}. | ||
*/ | ||
public static void printEntities(PrintStream out, List<Entity> entities) { | ||
if (entities == null || entities.size() == 0) { | ||
out.println("No entities found."); | ||
return; | ||
} | ||
out.printf("Found %d entit%s.\n", entities.size(), entities.size() == 1 ? "y" : "ies"); | ||
for (Entity entity : entities) { | ||
out.printf("%s\n", entity.getName()); | ||
out.printf("\tSalience: %.3f\n", entity.getSalience()); | ||
out.printf("\tType: %s\n", entity.getType()); | ||
if (entity.getMetadata() != null) { | ||
for (Map.Entry<String, String> metadata : entity.getMetadata().entrySet()) { | ||
out.printf("\tMetadata: %s = %s\n", metadata.getKey(), metadata.getValue()); | ||
} | ||
} | ||
} | ||
} | ||
|
||
/** | ||
* Print the Sentiment {@code sentiment}. | ||
*/ | ||
public static void printSentiment(PrintStream out, Sentiment sentiment) { | ||
if (sentiment == null) { | ||
out.println("No sentiment found"); | ||
return; | ||
} | ||
out.println("Found sentiment."); | ||
out.printf("\tMagnitude: %.3f\n", sentiment.getMagnitude()); | ||
out.printf("\tPolarity: %.3f\n", sentiment.getPolarity()); | ||
} | ||
|
||
public static void printSyntax(PrintStream out, List<Token> tokens) { | ||
if (tokens == null || tokens.size() == 0) { | ||
out.println("No syntax found"); | ||
return; | ||
} | ||
out.printf("Found %d token%s.\n", tokens.size(), tokens.size() == 1 ? "" : "s"); | ||
for (Token token : tokens) { | ||
out.println("TextSpan"); | ||
out.printf("\tText: %s\n", token.getText().getContent()); | ||
out.printf("\tBeginOffset: %d\n", token.getText().getBeginOffset()); | ||
out.printf("Lemma: %s\n", token.getLemma()); | ||
out.printf("PartOfSpeechTag: %s\n", token.getPartOfSpeech().getTag()); | ||
out.println("DependencyEdge"); | ||
out.printf("\tHeadTokenIndex: %d\n", token.getDependencyEdge().getHeadTokenIndex()); | ||
out.printf("\tLabel: %s\n", token.getDependencyEdge().getLabel()); | ||
} | ||
} | ||
|
||
/** | ||
* Connects to the Natural Language API using Application Default Credentials. | ||
*/ | ||
public static CloudNaturalLanguageAPI getLanguageService() | ||
throws IOException, GeneralSecurityException { | ||
GoogleCredential credential = | ||
GoogleCredential.getApplicationDefault().createScoped(CloudNaturalLanguageAPIScopes.all()); | ||
JsonFactory jsonFactory = JacksonFactory.getDefaultInstance(); | ||
return new CloudNaturalLanguageAPI.Builder( | ||
GoogleNetHttpTransport.newTrustedTransport(), | ||
jsonFactory, new HttpRequestInitializer() { | ||
@Override | ||
public void initialize(HttpRequest request) throws IOException { | ||
credential.initialize(request); | ||
} | ||
}) | ||
.setApplicationName(APPLICATION_NAME) | ||
.build(); | ||
} | ||
|
||
private final CloudNaturalLanguageAPI languageApi; | ||
|
||
/** | ||
* Constructs a {@link Analyze} which connects to the Cloud Natural Language API. | ||
*/ | ||
public Analyze(CloudNaturalLanguageAPI languageApi) { | ||
this.languageApi = languageApi; | ||
} | ||
|
||
/** | ||
* Gets {@link Entity}s from the string {@code text}. | ||
*/ | ||
public List<Entity> analyzeEntities(String text) throws IOException { | ||
AnalyzeEntitiesRequest request = | ||
new AnalyzeEntitiesRequest() | ||
.setDocument(new Document().setContent(text).setType("PLAIN_TEXT")) | ||
.setEncodingType("UTF16"); | ||
CloudNaturalLanguageAPI.Documents.AnalyzeEntities analyze = | ||
languageApi.documents().analyzeEntities(request); | ||
|
||
AnalyzeEntitiesResponse response = analyze.execute(); | ||
return response.getEntities(); | ||
} | ||
|
||
/** | ||
* Gets {@link Sentiment} from the string {@code text}. | ||
*/ | ||
public Sentiment analyzeSentiment(String text) throws IOException { | ||
AnalyzeSentimentRequest request = | ||
new AnalyzeSentimentRequest() | ||
.setDocument(new Document().setContent(text).setType("PLAIN_TEXT")); | ||
CloudNaturalLanguageAPI.Documents.AnalyzeSentiment analyze = | ||
languageApi.documents().analyzeSentiment(request); | ||
|
||
AnalyzeSentimentResponse response = analyze.execute(); | ||
return response.getDocumentSentiment(); | ||
} | ||
|
||
/** | ||
* Gets {@link Token}s from the string {@code text}. | ||
*/ | ||
public List<Token> analyzeSyntax(String text) throws IOException { | ||
AnnotateTextRequest request = | ||
new AnnotateTextRequest() | ||
.setDocument(new Document().setContent(text).setType("PLAIN_TEXT")) | ||
.setFeatures(new Features().setExtractSyntax(true)) | ||
.setEncodingType("UTF16"); | ||
CloudNaturalLanguageAPI.Documents.AnnotateText analyze = | ||
languageApi.documents().annotateText(request); | ||
|
||
AnnotateTextResponse response = analyze.execute(); | ||
return response.getTokens(); | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This really isn't necessary in most instances. But I won't block on it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I dunno - I feel like it's the most failsafe way to do things. The other methods of getting credentials have too many caveats attached to them :-/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a huge fan of gcloud for credentials and DefaultAuth.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here my (possibly outdated) understanding:
gcloud
auth login
which works sometimes but fails subtly because it uses user auth on the cloudsdktool project instead of a service account that's attached to the project. Also you'll have to swap this out with a service account anyway once you go to production, if you're not on GCE/GAE.auth activate-service-account
which requires downloading a service account anyway, and what happens in your production scripts if you're not on GCE/GAE? I suppose you could have your machine run thegcloud
command as part of setup? Seems like more overhead than setting an env variable..gcloud
on every machine you run the code on (unless you're on GAE/GCE)I assume by DefaultAuth you mean the default service account on GAE/GCE? Which is fine, unless you're not on GAE/GCE, in which case we have to mention that and explain what to do in that case, so it doesn't save any explanation space (though yes - if you are running on GAE/GCE, you can just skip that paragraph).
Point being - it's fine, but there's a lot of explaining and decision points around whether you're on GAE/GCE, that folks probably don't care about when they're looking at a sample like this, so it's simpler to just have one not-too-complex explanation that works in all instances :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
gcloud beta auth application-default
is the latest hotness.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately doesn't seem to work :(
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should let Vijay know.