Skip to content
This repository has been archived by the owner on Oct 3, 2023. It is now read-only.

docs(samples): Refactoring product package (Import) (new) #480

Merged
merged 29 commits into from
Sep 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
c50efe7
Cherry-picked "Update README"
tetiana-karasova Mar 22, 2022
f643f83
Cherry-picked "the bash scripts are added"
t-karasova Mar 31, 2022
62f947d
Cherry-picked "Updated README"
t-karasova Apr 1, 2022
aa1ce1d
Update user_environment_setup.sh
t-karasova Apr 1, 2022
5ec159a
Update user_import_data_to_catalog.sh
t-karasova Apr 1, 2022
b308d9b
Cherry-picked "Updated README"
t-karasova Apr 5, 2022
2c90e8e
Refactoring product package (Import).
sborisenkox Apr 25, 2022
f0f490d
Remove init package.
sborisenkox Apr 25, 2022
fcb2411
Update README
dfirova Jul 1, 2022
eba6357
Removed assuredBreak.
dfirova Jul 14, 2022
c215629
PR fix: removed tests for setup/cleanup.
dfirova Jul 18, 2022
f6ad617
PR fix: fixed the deadline.
dfirova Jul 20, 2022
6274346
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Jul 27, 2022
dab57ce
PR fix: unique bucket name.
dfirova Aug 9, 2022
893e576
PR fix: unique bucket name.
dfirova Aug 9, 2022
5069936
PR fix: removed test.
dfirova Aug 9, 2022
f394e08
Merge branch 'main' into refactoring-products-import
dfirova Aug 11, 2022
24ab126
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Aug 11, 2022
61bcab2
Merge branch 'main' into refactoring-products-import
dfirova Aug 12, 2022
e7cdbb8
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Aug 12, 2022
fe0dcdf
Merge branch 'main' into refactoring-products-import
dfirova Aug 19, 2022
ca8ebda
Merge branch 'googleapis:main' into refactoring-products-import
dfirova Aug 25, 2022
9a7c57b
Merge branch 'main' into refactoring-products-import
dfirova Aug 25, 2022
10b8271
Merge remote-tracking branch 'origin/refactoring-products-import' int…
dfirova Aug 25, 2022
8b8b2b2
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Aug 26, 2022
135e0a7
PR fix: changed tests to be similar to events test.
dfirova Aug 31, 2022
7da8114
Merge remote-tracking branch 'origin/refactoring-products-import' int…
dfirova Aug 31, 2022
2ddf274
PR fix: returned test to previous state.
dfirova Sep 1, 2022
9ff7076
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Sep 8, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,6 @@ Samples are in the [`samples/`](https://github.com/googleapis/java-retail/tree/m
| Events Create Big Query Table | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/events/setup/EventsCreateBigQueryTable.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/events/setup/EventsCreateBigQueryTable.java) |
| Events Create Gcs Bucket | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/events/setup/EventsCreateGcsBucket.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/events/setup/EventsCreateGcsBucket.java) |
| Update User Events Json | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/events/setup/UpdateUserEventsJson.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/events/setup/UpdateUserEventsJson.java) |
| Create Test Resources | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/init/CreateTestResources.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/init/CreateTestResources.java) |
| Remove Test Resources | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/init/RemoveTestResources.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/init/RemoveTestResources.java) |
| Add Fulfillment Places | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/product/AddFulfillmentPlaces.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/product/AddFulfillmentPlaces.java) |
| Create Product | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/product/CreateProduct.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/product/CreateProduct.java) |
| Crud Product | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/product/CrudProduct.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/product/CrudProduct.java) |
Expand All @@ -101,6 +99,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/java-retail/tree/m
| Update Product | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/product/UpdateProduct.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/product/UpdateProduct.java) |
| Products Create Bigquery Table | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/product/setup/ProductsCreateBigqueryTable.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/product/setup/ProductsCreateBigqueryTable.java) |
| Products Create Gcs Bucket | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/product/setup/ProductsCreateGcsBucket.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/product/setup/ProductsCreateGcsBucket.java) |
| Remove Products Resources | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/product/setup/RemoveProductsResources.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/product/setup/RemoveProductsResources.java) |
| Search Simple Query | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/search/SearchSimpleQuery.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/search/SearchSimpleQuery.java) |
| Search With Boost Spec | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/search/SearchWithBoostSpec.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/search/SearchWithBoostSpec.java) |
| Search With Facet Spec | [source code](https://github.com/googleapis/java-retail/blob/main/samples/interactive-tutorials/src/main/java/search/SearchWithFacetSpec.java) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/java-retail&page=editor&open_in_editor=samples/interactive-tutorials/src/main/java/search/SearchWithFacetSpec.java) |
Expand Down

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -33,36 +33,30 @@
import com.google.longrunning.Operation;
import com.google.longrunning.OperationsClient;
import java.io.IOException;
import java.util.concurrent.TimeUnit;

public class ImportProductsBigQueryTable {

public static void main(String[] args) throws IOException, InterruptedException {
// TODO(developer): Replace these variables before running the sample.
String projectId = ServiceOptions.getDefaultProjectId();
String branchName =
String.format(
"projects/%s/locations/global/catalogs/default_catalog/branches/0", projectId);
String datasetId = "products";
String tableId = "products";
// TO CHECK ERROR HANDLING USE THE TABLE WITH INVALID PRODUCTS:
// TABLE_ID = "products_some_invalid"
String dataSchema = "product";
// TRY THE FULL RECONCILIATION MODE HERE:
ReconciliationMode reconciliationMode = ReconciliationMode.INCREMENTAL;
// tableId = "products_some_invalid"

ImportProductsRequest importBigQueryRequest =
getImportProductsBigQueryRequest(
reconciliationMode, projectId, datasetId, tableId, dataSchema, branchName);
waitForOperationCompletion(importBigQueryRequest);
importProductsFromBigQuery(projectId, branchName, datasetId, tableId);
}

public static ImportProductsRequest getImportProductsBigQueryRequest(
ReconciliationMode reconciliationMode,
String projectId,
String datasetId,
String tableId,
String dataSchema,
String branchName) {
public static void importProductsFromBigQuery(
String projectId, String branchName, String datasetId, String tableId)
throws IOException, InterruptedException {
// TRY THE FULL RECONCILIATION MODE HERE:
ReconciliationMode reconciliationMode = ReconciliationMode.INCREMENTAL;
String dataSchema = "product";

BigQuerySource bigQuerySource =
BigQuerySource.newBuilder()
.setProjectId(projectId)
Expand All @@ -82,30 +76,29 @@ public static ImportProductsRequest getImportProductsBigQueryRequest(
.build();
System.out.printf("Import products from big query table request: %s%n", importRequest);

return importRequest;
}

public static void waitForOperationCompletion(ImportProductsRequest importRequest)
throws IOException, InterruptedException {
// Initialize client that will be used to send requests. This client only
// needs to be created once, and can be reused for multiple requests. After
// completing all of your requests, call the "close" method on the client to
// safely clean up any remaining background resources.
try (ProductServiceClient serviceClient = ProductServiceClient.create()) {
String operationName = serviceClient.importProductsCallable().call(importRequest).getName();
System.out.printf("OperationName = %s\n", operationName);
System.out.printf("OperationName = %s%n", operationName);

OperationsClient operationsClient = serviceClient.getOperationsClient();
Operation operation = operationsClient.getOperation(operationName);

while (!operation.getDone()) {
// Keep polling the operation periodically until the import task is done.
Thread.sleep(30_000);
TimeUnit.SECONDS.sleep(30);
operation = operationsClient.getOperation(operationName);
}

if (operation.hasMetadata()) {
ImportMetadata metadata = operation.getMetadata().unpack(ImportMetadata.class);
System.out.printf(
"Number of successfully imported products: %s\n", metadata.getSuccessCount());
"Number of successfully imported products: %s%n", metadata.getSuccessCount());
System.out.printf(
"Number of failures during the importing: %s\n", metadata.getFailureCount());
"Number of failures during the importing: %s%n", metadata.getFailureCount());
}

if (operation.hasResponse()) {
Expand Down
Loading