From 7cc5fa82a047ddec979d5af2f6f0c4a0d70b583a Mon Sep 17 00:00:00 2001 From: Rakhat <51375666+zethuman@users.noreply.github.com> Date: Tue, 4 Apr 2023 20:03:51 +0600 Subject: [PATCH 1/7] Add markdown linter (#261) * Add linter for markdown files Signed-off-by: Rakhat Zhuman * Updated CHANGELOG.md Signed-off-by: Rakhat Zhuman * Add make-script and update DEVELOPER_GUIDE.md Signed-off-by: Rakhat Zhuman * Updated linter options Signed-off-by: Rakhat Zhuman * Fixed mmarkdown files format Signed-off-by: Rakhat Zhuman --------- Signed-off-by: Rakhat Zhuman --- .github/workflows/lint.yml | 11 ++++++ ADMINS.md | 14 +++---- CHANGELOG.md | 3 +- DEVELOPER_GUIDE.md | 29 ++++++++------ MAINTAINERS.md | 24 ++++++------ Makefile | 37 ++++++++++++++++++ README.md | 8 +--- UPGRADING.md | 12 +++--- USER_GUIDE.md | 77 ++++++++++++++++++++------------------ 9 files changed, 134 insertions(+), 81 deletions(-) diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index ed6e387e4..7422aee3d 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -27,3 +27,14 @@ jobs: shell: bash - run: go version - run: make lint + + prettify: + name: Prettify + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + with: { fetch-depth: 1 } + - name: Install dependencies + run: npm i -g npm@8 && npm i -g prettier + - name: Check markdown files + run: prettier --prose-wrap never --check **/*.md diff --git a/ADMINS.md b/ADMINS.md index 1ec1f9e4b..f9220c93b 100644 --- a/ADMINS.md +++ b/ADMINS.md @@ -11,13 +11,13 @@ This document explains who the admins are (see below), what they do in this repo ## Current Admins -| Admin | GitHub ID | Affiliation | -| ----------------------- | ------------------------------------------- | ----------- | -| Charlotte | [CEHENKLE](https://github.com/CEHENKLE) | Amazon | -| Henri Yandell | [hyandell](https://github.com/hyandell) | Amazon | -| Jack Mazanec | [jmazanec15](https://github.com/jmazanec15) | Amazon | -| Vamshi Vijay Nakkirtha | [vamshin](https://github.com/vamshin) | Amazon | -| Vijayan Balasubramanian | [VijayanB](https://github.com/VijayanB) | Amazon | +| Admin | GitHub ID | Affiliation | +| --- | --- | --- | +| Charlotte | [CEHENKLE](https://github.com/CEHENKLE) | Amazon | +| Henri Yandell | [hyandell](https://github.com/hyandell) | Amazon | +| Jack Mazanec | [jmazanec15](https://github.com/jmazanec15) | Amazon | +| Vamshi Vijay Nakkirtha | [vamshin](https://github.com/vamshin) | Amazon | +| Vijayan Balasubramanian | [VijayanB](https://github.com/VijayanB) | Amazon | ## Admin Responsibilities diff --git a/CHANGELOG.md b/CHANGELOG.md index 4718a1889..5dcc0ad5b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) - Adds `Err()` function to Response for detailed errors ([#246](https://github.com/opensearch-project/opensearch-go/pull/246)) - Adds Point In Time API ([#253](https://github.com/opensearch-project/opensearch-go/pull/253)) - Adds InfoResp type ([#253](https://github.com/opensearch-project/opensearch-go/pull/253)) +- Adds markdown linter ([#261](https://github.com/opensearch-project/opensearch-go/pull/261)) - Adds testcases to check upsert functionality ([#207](https://github.com/opensearch-project/opensearch-go/issues/207)) - Added @Jakob3xD to co-maintainers ([#270](https://github.com/opensearch-project/opensearch-go/pull/270)) @@ -44,4 +45,4 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) ### Security -[Unreleased]: https://github.com/opensearch-project/opensearch-go/compare/2.1...HEAD \ No newline at end of file +[Unreleased]: https://github.com/opensearch-project/opensearch-go/compare/2.1...HEAD diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index c4279a2f2..99eb3ac87 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -7,6 +7,8 @@ - [Unit Testing](#unit-testing) - [Integration Testing](#integration-testing) - [Execute integration tests from your terminal](#execute-integration-tests-from-your-terminal) + - [Lint](#lint) + - [Markdown lint](#markdown-lint) - [Use an Editor](#use-an-editor) - [GoLand](#goland) @@ -18,8 +20,7 @@ So you want to contribute code to the OpenSearch Go Client? Excellent! We're gla ### Git Clone OpenSearch Go Client Repository -Fork [opensearch-project/opensearch-go](https://github.com/opensearch-project/opensearch-go) and clone locally, -e.g. `git clone https://github.com/[your username]/opensearch-go.git`. +Fork [opensearch-project/opensearch-go](https://github.com/opensearch-project/opensearch-go) and clone locally, e.g. `git clone https://github.com/[your username]/opensearch-go.git`. ### Install Prerequisites @@ -48,13 +49,9 @@ go test -v -run TestName; ### Integration Testing -In order to test opensearch-go client, you need a running OpenSearch cluster. You can use Docker to accomplish this. -The [Docker Compose file](.ci/opensearch/docker-compose.yml) supports the ability to run integration tests for the project in local environments. -If you have not installed docker-compose, you can install it from this [link](https://docs.docker.com/compose/install/). +In order to test opensearch-go client, you need a running OpenSearch cluster. You can use Docker to accomplish this. The [Docker Compose file](.ci/opensearch/docker-compose.yml) supports the ability to run integration tests for the project in local environments. If you have not installed docker-compose, you can install it from this [link](https://docs.docker.com/compose/install/). -In order to differentiate unit tests from integration tests, Go has a built-in mechanism for allowing you to logically separate your tests -with [build tags](https://pkg.go.dev/cmd/go#hdr-Build_constraints). The build tag needs to be placed as close to the top of the file as possible, and must have a blank line beneath it. -Hence, create all integration tests with build tag 'integration'. +In order to differentiate unit tests from integration tests, Go has a built-in mechanism for allowing you to logically separate your tests with [build tags](https://pkg.go.dev/cmd/go#hdr-Build_constraints). The build tag needs to be placed as close to the top of the file as possible, and must have a blank line beneath it. Hence, create all integration tests with build tag 'integration'. #### Execute integration tests from your terminal @@ -71,6 +68,18 @@ Hence, create all integration tests with build tag 'integration'. make cluster.stop cluster.clean ``` +## Lint + +To keep all the code in a certain uniform format, it was decided to use some writing rules. If you wrote something wrong, it's okay, you can simply run the script to check the necessary files, and optionally format the content. But keep in mind that all these checks are repeated on the pipeline, so it's better to check locally. + +### Markdown lint + +To check the markdown files, run the following command: + +``` +make lint.markdown +``` + ## Use an Editor ### GoLand @@ -80,6 +89,4 @@ You can import the OpenSearch project into GoLand as follows: 1. Select **File | Open** 2. In the subsequent dialog navigate to the ~/go/src/opensearch-go and click **Open** -After you have opened your project, you need to specify the location of the Go SDK. -You can either specify a local path to the SDK or download it. To set the Go SDK, navigate to **Go | GOROOT** and -set accordingly. +After you have opened your project, you need to specify the location of the Go SDK. You can either specify a local path to the SDK or download it. To set the Go SDK, navigate to **Go | GOROOT** and set accordingly. diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 8a62409d4..e68e93297 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -4,18 +4,18 @@ This document contains a list of maintainers in this repo. See [opensearch-proje ## Current Maintainers -| Maintainer | GitHub ID | Affiliation | -| ----------------------- | ----------------------------------------- | ------------------- | -| Daniel Doubrovkine | [dblock](https://github.com/dblock) | Amazon | -| Jakob Hahn | [Jakob3xD](https://github.com/Jakob3xD) | Hetzner Online GmbH | -| Vacha Shah | [VachaShah](https://github.com/VachaShah) | Amazon | -| Vijayan Balasubramanian | [VijayanB](https://github.com/VijayanB) | Amazon | +| Maintainer | GitHub ID | Affiliation | +| --- | --- | --- | +| Daniel Doubrovkine | [dblock](https://github.com/dblock) | Amazon | +| Jakob Hahn | [Jakob3xD](https://github.com/Jakob3xD) | Hetzner Online GmbH | +| Vacha Shah | [VachaShah](https://github.com/VachaShah) | Amazon | +| Vijayan Balasubramanian | [VijayanB](https://github.com/VijayanB) | Amazon | ## Emeritus -| Maintainer | GitHub ID | Affiliation | -| ---------------------- | ------------------------------------------- | ----------- | -| Jack Mazanec | [jmazanec15](https://github.com/jmazanec15) | Amazon | -| Rob Cowart | [robcowart](https://github.com/robcowart) | ElastiFlow | -| Sven Cowart | [svencowart](https://github.com/svencowart) | | -| Vamshi Vijay Nakkirtha | [vamshin](https://github.com/vamshin) | Amazon | +| Maintainer | GitHub ID | Affiliation | +| --- | --- | --- | +| Jack Mazanec | [jmazanec15](https://github.com/jmazanec15) | Amazon | +| Rob Cowart | [robcowart](https://github.com/robcowart) | ElastiFlow | +| Sven Cowart | [svencowart](https://github.com/svencowart) | | +| Vamshi Vijay Nakkirtha | [vamshin](https://github.com/vamshin) | Amazon | diff --git a/Makefile b/Makefile index 2548e4a95..ade9b9667 100644 --- a/Makefile +++ b/Makefile @@ -66,6 +66,42 @@ lint: ## Run lint on the package cd "internal/build/" && go mod tidy && go mod download && go vet ./...; \ } +package := "prettier" +lint.markdown: + @printf "\033[2m→ Checking node installed...\033[0m\n" + if type node > /dev/null 2>&1 && which node > /dev/null 2>&1 ; then \ + node -v; \ + echo -e "\033[33m Node is installed, continue...\033[0m\n"; \ + else \ + echo -e "\033[31m Please install node\033[0m\n"; \ + exit 1; \ + fi + @printf "\033[2m→ Checking npm installed...\033[0m\n" + if type npm > /dev/null 2>&1 && which npm > /dev/null 2>&1 ; then \ + npm -v; \ + echo -e "\033[33m NPM is installed, continue...\033[0m\n"; \ + else \ + echo -e "\033[31m Please install npm\033[0m\n"; \ + exit 1; \ + fi + @printf "\033[2m→ Checking $(package) installed...\033[0m\n" + if [ `npm list -g | grep -c $(package)` -eq 0 -o ! -d node_module ]; then \ + echo -e "\033[33m Installing $(package)...\033[0m"; \ + npm install -g $(package) --no-shrinkwrap; \ + fi + @printf "\033[2m→ Running markdown lint...\033[0m\n" + if npx $(package) --prose-wrap never --check **/*.md; [[ $$? -ne 0 ]]; then \ + echo -e "\033[32m→ Found invalid files. Want to auto-format invalid files? (y/n) \033[0m"; \ + read RESP; \ + if [[ $$RESP = "y" || $$RESP = "Y" ]]; then \ + echo -e "\033[33m Formatting...\033[0m"; \ + npx $(package) --prose-wrap never --write **/*.md; \ + echo -e "\033[34m \nAll invalid files are formatted\033[0m"; \ + else \ + echo -e "\033[33m Unfortunately you are cancelled auto fixing. But we will definitely fix it in the pipeline\033[0m"; \ + fi \ + fi + backport: ## Backport one or more commits from main into version branches ifeq ($(origin commits), undefined) @@ -196,3 +232,4 @@ help: ## Display help .DEFAULT_GOAL := help .PHONY: help backport cluster cluster.clean coverage godoc lint release test test-bench test-integ test-unit linters linters.install +.SILENT: lint.markdown diff --git a/README.md b/README.md index 8161dfd43..bd0260fc6 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,4 @@ -[![Go Reference](https://pkg.go.dev/badge/github.com/opensearch-project/opensearch-go.svg)](https://pkg.go.dev/github.com/opensearch-project/opensearch-go/v2) -[![Build](https://github.com/opensearch-project/opensearch-go/actions/workflows/lint.yml/badge.svg)](https://github.com/opensearch-project/opensearch-go/actions/workflows/lint.yml) -[![Unit](https://github.com/opensearch-project/opensearch-go/actions/workflows/test-unit.yml/badge.svg)](https://github.com/opensearch-project/opensearch-go/actions/workflows/test-unit.yml) -[![Integration](https://github.com/opensearch-project/opensearch-go/actions/workflows/test-integration.yml/badge.svg)](https://github.com/opensearch-project/opensearch-go/actions/workflows/test-integration.yml) -[![codecov](https://codecov.io/gh/opensearch-project/opensearch-go/branch/main/graph/badge.svg?token=MI9g3KYHVx)](https://codecov.io/gh/opensearch-project/opensearch-go) -[![Chat](https://img.shields.io/badge/chat-on%20forums-blue)](https://discuss.opendistrocommunity.dev/c/clients/) -![PRs welcome!](https://img.shields.io/badge/PRs-welcome!-success) +[![Go Reference](https://pkg.go.dev/badge/github.com/opensearch-project/opensearch-go.svg)](https://pkg.go.dev/github.com/opensearch-project/opensearch-go/v2) [![Build](https://github.com/opensearch-project/opensearch-go/actions/workflows/lint.yml/badge.svg)](https://github.com/opensearch-project/opensearch-go/actions/workflows/lint.yml) [![Unit](https://github.com/opensearch-project/opensearch-go/actions/workflows/test-unit.yml/badge.svg)](https://github.com/opensearch-project/opensearch-go/actions/workflows/test-unit.yml) [![Integration](https://github.com/opensearch-project/opensearch-go/actions/workflows/test-integration.yml/badge.svg)](https://github.com/opensearch-project/opensearch-go/actions/workflows/test-integration.yml) [![codecov](https://codecov.io/gh/opensearch-project/opensearch-go/branch/main/graph/badge.svg?token=MI9g3KYHVx)](https://codecov.io/gh/opensearch-project/opensearch-go) [![Chat](https://img.shields.io/badge/chat-on%20forums-blue)](https://discuss.opendistrocommunity.dev/c/clients/) ![PRs welcome!](https://img.shields.io/badge/PRs-welcome!-success) ![OpenSearch logo](OpenSearch.svg) diff --git a/UPGRADING.md b/UPGRADING.md index 5981282ff..d09583b4d 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -1,7 +1,7 @@ - [Upgrading Opensearch GO Client](#upgrading-opensearch-go-client) - - [Upgraading to >= 3.0.0](#upgrading-to->=-3.0.0) - - [opensearchapi](#opensearchapi-snapshot-delete) - - [opensearchapi](#opensearchapi-error-handling) + - [Upgraading to >= 3.0.0](#upgrading-to->=-3.0.0) + - [opensearchapi](#opensearchapi-snapshot-delete) + - [opensearchapi](#opensearchapi-error-handling) # Upgrading Opensearch GO Client @@ -50,11 +50,10 @@ reqSnapshots := &opensearchapi.SnapshotDeleteRequest{ With opensearch-go >= 3.0.0 opensearchapi responses are now checked for errors. Checking for errors twice is no longer needed. -Prior versions only returned an error if the request failed to execute. For example if the client can't reach the server or the TLS handshake failed. -With opensearch-go >= 3.0.0 each opensearchapi requests will return an error if the response http status code is > 299. -The error can be parsed into the new `opensearchapi.Error` type by using `errors.As` to match for exceptions and get a more detailed view. +Prior versions only returned an error if the request failed to execute. For example if the client can't reach the server or the TLS handshake failed. With opensearch-go >= 3.0.0 each opensearchapi requests will return an error if the response http status code is > 299. The error can be parsed into the new `opensearchapi.Error` type by using `errors.As` to match for exceptions and get a more detailed view. Before 3.0.0: + ```go createIndex := opensearchapi.IndicesCreateRequest{ Index: IndexName, @@ -94,4 +93,3 @@ if err != nil { } } ``` - diff --git a/USER_GUIDE.md b/USER_GUIDE.md index db4a128cd..da129b133 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -1,21 +1,18 @@ - [User Guide](#user-guide) - - [Example](#example) - - [Amazon OpenSearch Service](#amazon-opensearch-service) - - [AWS SDK V1](#aws-sdk-v1) - - [AWS SDK V2](#aws-sdk-v2) - - [Data Streams API](#data-streams-api) - - [Create Data Stream](#create-data-streams) - - [Delete Data Stream](#delete-data-streams) - - [Get All Data Streams](#get-data-streams) - - [Get Specific Data Stream](#get-specific-data-streams) - - [Get Specific Data Stream Stats](#get-specific-data-streams-stats) + - [Example](#example) + - [Amazon OpenSearch Service](#amazon-opensearch-service) - [AWS SDK V1](#aws-sdk-v1) - [AWS SDK V2](#aws-sdk-v2) + - [Data Streams API](#data-streams-api) + - [Create Data Stream](#create-data-streams) + - [Delete Data Stream](#delete-data-streams) + - [Get All Data Streams](#get-data-streams) + - [Get Specific Data Stream](#get-specific-data-streams) + - [Get Specific Data Stream Stats](#get-specific-data-streams-stats) # User Guide ## Example -In the example below, we create a client, an index with non-default settings, insert a document to the index, -search for the document, delete the document and finally delete the index. +In the example below, we create a client, an index with non-default settings, insert a document to the index, search for the document, delete the document and finally delete the index. ```go package main @@ -171,26 +168,17 @@ func example() error { ## Amazon OpenSearch Service -Before starting, we strongly recommend reading the full AWS documentation regarding using IAM credentials to sign -requests to OpenSearch APIs. -See [Identity and Access Management in Amazon OpenSearch Service.](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html) +Before starting, we strongly recommend reading the full AWS documentation regarding using IAM credentials to sign requests to OpenSearch APIs. See [Identity and Access Management in Amazon OpenSearch Service.](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html) -> Even if you configure a completely open resource-based access policy, all requests to the OpenSearch Service -> configuration API must be signed. If your policies specify IAM users or roles, requests to the OpenSearch APIs also -> must -> be signed using AWS Signature Version 4. +> Even if you configure a completely open resource-based access policy, all requests to the OpenSearch Service configuration API must be signed. If your policies specify IAM users or roles, requests to the OpenSearch APIs also must be signed using AWS Signature Version 4. > > See [Managed Domains signing-service requests.](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html#managedomains-signing-service-requests) -Depending on the version of AWS SDK used, import the v1 or v2 request signer from `signer/aws` or `signer/awsv2` -respectively. -Both signers are equivalent in their functionality, they provide AWS Signature Version 4 (SigV4). +Depending on the version of AWS SDK used, import the v1 or v2 request signer from `signer/aws` or `signer/awsv2` respectively. Both signers are equivalent in their functionality, they provide AWS Signature Version 4 (SigV4). -To read more about SigV4 -see [Signature Version 4 signing process](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html) +To read more about SigV4 see [Signature Version 4 signing process](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html) -Here are some Go samples that show how to sign each OpenSearch request and automatically search for AWS credentials from -the ~/.aws folder or environment variables: +Here are some Go samples that show how to sign each OpenSearch request and automatically search for AWS credentials from the ~/.aws folder or environment variables: #### AWS SDK V1 @@ -344,7 +332,8 @@ func getCredentialProvider(accessKey, secretAccessKey, token string) aws.Credent ### Create Data Streams - - Create new client +- Create new client + ``` client, err := opensearch.NewDefaultClient() if err != nil { @@ -352,7 +341,7 @@ if err != nil { } ``` - - Create template index +- Create template index ``` iPut := opensearchapi.IndicesPutIndexTemplateRequest{ @@ -365,7 +354,8 @@ iPut := opensearchapi.IndicesPutIndexTemplateRequest{ iPutResponse, err := iPut.Do(context.Background(), client) ``` - - Prepare request object +- Prepare request object + ``` es := opensearchapi.IndicesCreateDataStreamRequest{ Name: "demo-name", @@ -378,7 +368,8 @@ es := opensearchapi.IndicesCreateDataStreamRequest{ } ``` - - Execute request +- Execute request + ``` res, err := es.Do(context.TODO(), client) if err != nil { @@ -387,7 +378,8 @@ if err != nil { } ``` - - Try to read response +- Try to read response + ``` defer res.Body.Close() body, err := ioutil.ReadAll(res.Body) @@ -401,7 +393,8 @@ fmt.Println("Response Headers: ", res.Header) fmt.Println("Response Body: ", string(body)) ``` - - Successfully created data stream +- Successfully created data stream + ``` Response Status Code: 200 Response Headers: map[Content-Length:[28] Content-Type:[application/json; charset=UTF-8]] @@ -410,8 +403,9 @@ Response Body: {"acknowledged" : true} ### Delete Data Streams - - Create new client as previous example - - Prepare request object +- Create new client as previous example +- Prepare request object + ``` opensearchapi.IndicesDeleteDataStreamRequest{ Name: "demo-name", @@ -423,9 +417,11 @@ opensearchapi.IndicesDeleteDataStreamRequest{ }, } ``` + - Execute request as previous example - Try to read response as previous example -- Successfully deleted data stream +- Successfully deleted data stream + ``` Response Status Code: 200 Response Headers: map[Content-Length:[28] Content-Type:[application/json; charset=UTF-8]] @@ -436,6 +432,7 @@ Response Body: {"acknowledged" : true} - Create new client as previous example - Prepare request object + ``` r := opensearchapi.IndicesGetDataStreamRequest{ Pretty: true, @@ -446,9 +443,11 @@ r := opensearchapi.IndicesGetDataStreamRequest{ }, } ``` + - Execute request as previous example - Try to read response as previous example - Successfully retrieved data streams + ``` Response Status Code: 200 Response Headers: map[Content-Length:[28] Content-Type:[application/json; charset=UTF-8]] @@ -459,6 +458,7 @@ Response Body: {"data_streams":[{"name":"demo-name","timestamp_field":{"n - Create new client as previous example - Prepare request object + ``` r := opensearchapi.IndicesGetDataStreamRequest{ Name: "demo-name", @@ -470,9 +470,11 @@ r := opensearchapi.IndicesGetDataStreamRequest{ }, } ``` + - Execute request as previous example - Try to read response as previous example - Successfully retrieved data stream + ``` Response Status Code: 200 Response Headers: map[Content-Length:[28] Content-Type:[application/json; charset=UTF-8]] @@ -483,6 +485,7 @@ Response Body: {"data_streams":[{"name":"demo-name","timestamp_field":{"n - Create new client as as previous example - Prepare request object + ``` r := opensearchapi.IndicesGetDataStreamStatsRequest{ Name: "demo-name", @@ -494,11 +497,13 @@ r := opensearchapi.IndicesGetDataStreamStatsRequest{ }, } ``` + - Execute request as previous example - Try to read response as previous example - Successfully retrieved data stream stats + ``` Response Status Code: 200 Response Headers: map[Content-Length:[28] Content-Type:[application/json; charset=UTF-8]] Response Body: {"_shards":{"total":2,"successful":1,"failed":0},"data_stream_count":1,"backing_indices":1,"total_store_size":"208b","total_store_size_bytes":208,"data_streams":[{"data_stream":"demo-name","backing_indices":1,"store_size":"208b","store_size_bytes":208,"maximum_timestamp":0}]} -``` \ No newline at end of file +``` From 8f127f0642b9081937bea2d64260801d26a54563 Mon Sep 17 00:00:00 2001 From: "Daniel (dB.) Doubrovkine" Date: Tue, 4 Apr 2023 13:07:47 -0400 Subject: [PATCH 2/7] Added Jakob3xD to CODEOWNERS, removed @svencowart. (#272) Signed-off-by: dblock --- .github/CODEOWNERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 890116d5e..d22bb719d 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1,2 +1,2 @@ # This should match the team set up in https://github.com/orgs/opensearch-project/teams and include any additional contributors -* @dblock @VijayanB @svencowart @VachaShah \ No newline at end of file +* @dblock @VijayanB @VachaShah @Jakob3xD \ No newline at end of file From 926535da9ecb966f5cf7ecb593569aec27f9f9e3 Mon Sep 17 00:00:00 2001 From: Jakob Date: Mon, 10 Apr 2023 23:01:48 +0200 Subject: [PATCH 3/7] opensearchapi: Fix handling of errors without error response body (#286) Signed-off-by: Jakob Hahn --- CHANGELOG.md | 3 +- opensearchapi/opensearchapi.error.go | 2 +- opensearchapi/opensearchapi.response.go | 3 +- .../opensearchapi_response_internal_test.go | 28 ++++++++++++++++--- 4 files changed, 29 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5dcc0ad5b..2ef5ba96a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,7 +22,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) - Adds InfoResp type ([#253](https://github.com/opensearch-project/opensearch-go/pull/253)) - Adds markdown linter ([#261](https://github.com/opensearch-project/opensearch-go/pull/261)) - Adds testcases to check upsert functionality ([#207](https://github.com/opensearch-project/opensearch-go/issues/207)) -- Added @Jakob3xD to co-maintainers ([#270](https://github.com/opensearch-project/opensearch-go/pull/270)) +- Adds @Jakob3xD to co-maintainers ([#270](https://github.com/opensearch-project/opensearch-go/pull/270)) ### Changed @@ -42,6 +42,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) - Renames the sequence number struct tag to `if_seq_no` to fix optimistic concurrency control ([#166](https://github.com/opensearch-project/opensearch-go/pull/166)) - Fixes `RetryOnConflict` on bulk indexer ([#215](https://github.com/opensearch-project/opensearch-go/pull/215)) - Corrects curl logging to emit the correct URL destination ([#101](https://github.com/opensearch-project/opensearch-go/pull/101)) +- Corrects handling of errors without an error response body ([#286](https://github.com/opensearch-project/opensearch-go/pull/286)) ### Security diff --git a/opensearchapi/opensearchapi.error.go b/opensearchapi/opensearchapi.error.go index cfaa155a3..5c5a30575 100644 --- a/opensearchapi/opensearchapi.error.go +++ b/opensearchapi/opensearchapi.error.go @@ -49,5 +49,5 @@ type RootCause struct { // Error returns a string. func (e *Error) Error() string { - return fmt.Sprintf("error: %s, status: %d", e.Err, e.Status) + return fmt.Sprintf("status: %d, type: %s, reason: %s, root_cause: %s", e.Status, e.Err.Type, e.Err.Reason, e.Err.RootCause) } diff --git a/opensearchapi/opensearchapi.response.go b/opensearchapi/opensearchapi.response.go index d4344ee48..0f4c561f4 100644 --- a/opensearchapi/opensearchapi.response.go +++ b/opensearchapi/opensearchapi.response.go @@ -33,6 +33,7 @@ import ( "io" "io/ioutil" "net/http" + "reflect" "strconv" "strings" ) @@ -120,7 +121,7 @@ func (r *Response) Err() error { } var e *Error err = json.Unmarshal(body, &e) - if err == nil { + if err == nil && !reflect.ValueOf(e.Err).IsZero() { return e } return fmt.Errorf("status: %d, error: %s", r.StatusCode, string(body)) diff --git a/opensearchapi/opensearchapi_response_internal_test.go b/opensearchapi/opensearchapi_response_internal_test.go index 57fd6a96b..0cdc7c61f 100644 --- a/opensearchapi/opensearchapi_response_internal_test.go +++ b/opensearchapi/opensearchapi_response_internal_test.go @@ -46,6 +46,7 @@ func TestAPIResponse(t *testing.T) { var ( body string res *Response + err error ) t.Run("String", func(t *testing.T) { @@ -108,16 +109,36 @@ func TestAPIResponse(t *testing.T) { t.Run("Error", func(t *testing.T) { res = &Response{StatusCode: 201} - if err := res.Err(); err != nil { + if err = res.Err(); err != nil { t.Errorf("Unexpected error for response: %s", res.Status()) } res = &Response{StatusCode: 403} - if err := res.Err(); err == nil { + if err = res.Err(); err == nil { t.Errorf("Expected error for response: %s", res.Status()) } + res = &Response{ + StatusCode: 404, + Body: io.NopCloser( + strings.NewReader(` + { + "_index":"index", + "_id":"2", + "matched":false + }`), + ), + } + err = res.Err() + if err == nil { + t.Errorf("Expected error for response: %s", res.Status()) + } + var errTest *Error + if errors.As(err, &errTest) { + t.Errorf("Expected error NOT to be of type opensearchapi.Error: %T", err) + } + res = &Response{ StatusCode: 400, Body: io.NopCloser( @@ -139,11 +160,10 @@ func TestAPIResponse(t *testing.T) { }`)), } - err := res.Err() + err = res.Err() if err == nil { t.Errorf("Expected error for response: %s", res.Status()) } - var errTest *Error if !errors.As(err, &errTest) { t.Errorf("Expected error to be of type opensearchapi.Error: %T", err) } From dae613e210fc31bf3d69c2a53f0941dc2246ac12 Mon Sep 17 00:00:00 2001 From: Vacha Shah Date: Tue, 11 Apr 2023 13:58:31 -0700 Subject: [PATCH 4/7] Add prettier formatting to dependabot prs (#297) Signed-off-by: Vacha Shah --- .github/workflows/dependabot_pr.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/dependabot_pr.yml b/.github/workflows/dependabot_pr.yml index 3acba188c..29898e581 100644 --- a/.github/workflows/dependabot_pr.yml +++ b/.github/workflows/dependabot_pr.yml @@ -33,6 +33,9 @@ jobs: uses: dangoslen/dependabot-changelog-helper@v2 with: version: 'Unreleased' + + - name: Format markdown files with prettier + run: prettier --prose-wrap never --write **/*.md - name: Commit the changes uses: stefanzweifel/git-auto-commit-action@v4 From e8de6f3169ec37840c8c01b1bd0a8b2890117703 Mon Sep 17 00:00:00 2001 From: Jakob Date: Tue, 11 Apr 2023 23:49:27 +0200 Subject: [PATCH 5/7] Install prettier dependency in workflow for dependabot (#299) Signed-off-by: Jakob Hahn --- .github/workflows/dependabot_pr.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/dependabot_pr.yml b/.github/workflows/dependabot_pr.yml index 29898e581..619665e06 100644 --- a/.github/workflows/dependabot_pr.yml +++ b/.github/workflows/dependabot_pr.yml @@ -34,6 +34,9 @@ jobs: with: version: 'Unreleased' + - name: Install dependencies + run: npm i -g npm@8 && npm i -g prettier + - name: Format markdown files with prettier run: prettier --prose-wrap never --write **/*.md From ccd4401e3821233f17d01d57d8f7ce641d2a4107 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 11 Apr 2023 15:02:21 -0700 Subject: [PATCH 6/7] Bump github.com/aws/aws-sdk-go from 1.44.235 to 1.44.241 (#298) * Bump github.com/aws/aws-sdk-go from 1.44.235 to 1.44.241 Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.235 to 1.44.241. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.235...v1.44.241) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] * Update changelog Signed-off-by: dependabot[bot] --------- Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] --- CHANGELOG.md | 2 +- go.mod | 2 +- go.sum | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2ef5ba96a..b7f39ed56 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) ### Dependencies -- Bumps `github.com/aws/aws-sdk-go` from 1.44.45 to 1.44.235 +- Bumps `github.com/aws/aws-sdk-go` from 1.44.45 to 1.44.241 - Bumps `github.com/aws/aws-sdk-go-v2` from 1.17.1 to 1.17.6 - Bumps `github.com/aws/aws-sdk-go-v2/config` from 1.18.8 to 1.18.19 - Bumps `github.com/stretchr/testify` from 1.8.0 to 1.8.2 diff --git a/go.mod b/go.mod index 6062a70c6..db1922dae 100644 --- a/go.mod +++ b/go.mod @@ -3,7 +3,7 @@ module github.com/opensearch-project/opensearch-go/v2 go 1.15 require ( - github.com/aws/aws-sdk-go v1.44.235 + github.com/aws/aws-sdk-go v1.44.241 github.com/aws/aws-sdk-go-v2 v1.17.7 github.com/aws/aws-sdk-go-v2/config v1.18.19 github.com/stretchr/testify v1.8.2 diff --git a/go.sum b/go.sum index e9d89d827..cf3a9352d 100644 --- a/go.sum +++ b/go.sum @@ -1,5 +1,5 @@ -github.com/aws/aws-sdk-go v1.44.235 h1:5MS1ZW1Pr27mmHFqqjuXYwGMlNTW/g6DqU5ekamPMeU= -github.com/aws/aws-sdk-go v1.44.235/go.mod h1:aVsgQcEevwlmQ7qHE9I3h+dtQgpqhFB+i8Phjh7fkwI= +github.com/aws/aws-sdk-go v1.44.241 h1:D3KycZq3HjhmjYGzvTcmX/Ztf/KNmsfTmdDuKdnzZKo= +github.com/aws/aws-sdk-go v1.44.241/go.mod h1:aVsgQcEevwlmQ7qHE9I3h+dtQgpqhFB+i8Phjh7fkwI= github.com/aws/aws-sdk-go-v2 v1.17.7 h1:CLSjnhJSTSogvqUGhIC6LqFKATMRexcxLZ0i/Nzk9Eg= github.com/aws/aws-sdk-go-v2 v1.17.7/go.mod h1:uzbQtefpm44goOPmdKyAlXSNcwlRgF3ePWVW6EtJvvw= github.com/aws/aws-sdk-go-v2/config v1.18.19 h1:AqFK6zFNtq4i1EYu+eC7lcKHYnZagMn6SW171la0bGw= From 486e5e9cffda6401a5f27e22fd62a6fc990e46a0 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 11 Apr 2023 15:19:13 -0700 Subject: [PATCH 7/7] Bump github.com/aws/aws-sdk-go-v2/config from 1.18.19 to 1.18.21 (#295) * Bump github.com/aws/aws-sdk-go-v2/config from 1.18.19 to 1.18.21 Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.18.19 to 1.18.21. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.18.19...config/v1.18.21) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] * Update changelog Signed-off-by: dependabot[bot] --------- Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] --- CHANGELOG.md | 2 +- go.mod | 4 ++-- go.sum | 44 ++++++++++++++++++++++---------------------- 3 files changed, 25 insertions(+), 25 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b7f39ed56..b1edbee9a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) - Bumps `github.com/aws/aws-sdk-go` from 1.44.45 to 1.44.241 - Bumps `github.com/aws/aws-sdk-go-v2` from 1.17.1 to 1.17.6 -- Bumps `github.com/aws/aws-sdk-go-v2/config` from 1.18.8 to 1.18.19 +- Bumps `github.com/aws/aws-sdk-go-v2/config` from 1.18.8 to 1.18.21 - Bumps `github.com/stretchr/testify` from 1.8.0 to 1.8.2 ### Added diff --git a/go.mod b/go.mod index db1922dae..7f6d222b4 100644 --- a/go.mod +++ b/go.mod @@ -4,8 +4,8 @@ go 1.15 require ( github.com/aws/aws-sdk-go v1.44.241 - github.com/aws/aws-sdk-go-v2 v1.17.7 - github.com/aws/aws-sdk-go-v2/config v1.18.19 + github.com/aws/aws-sdk-go-v2 v1.17.8 + github.com/aws/aws-sdk-go-v2/config v1.18.21 github.com/stretchr/testify v1.8.2 golang.org/x/net v0.7.0 // indirect ) diff --git a/go.sum b/go.sum index cf3a9352d..da831e0f9 100644 --- a/go.sum +++ b/go.sum @@ -1,27 +1,27 @@ github.com/aws/aws-sdk-go v1.44.241 h1:D3KycZq3HjhmjYGzvTcmX/Ztf/KNmsfTmdDuKdnzZKo= github.com/aws/aws-sdk-go v1.44.241/go.mod h1:aVsgQcEevwlmQ7qHE9I3h+dtQgpqhFB+i8Phjh7fkwI= -github.com/aws/aws-sdk-go-v2 v1.17.7 h1:CLSjnhJSTSogvqUGhIC6LqFKATMRexcxLZ0i/Nzk9Eg= -github.com/aws/aws-sdk-go-v2 v1.17.7/go.mod h1:uzbQtefpm44goOPmdKyAlXSNcwlRgF3ePWVW6EtJvvw= -github.com/aws/aws-sdk-go-v2/config v1.18.19 h1:AqFK6zFNtq4i1EYu+eC7lcKHYnZagMn6SW171la0bGw= -github.com/aws/aws-sdk-go-v2/config v1.18.19/go.mod h1:XvTmGMY8d52ougvakOv1RpiTLPz9dlG/OQHsKU/cMmY= -github.com/aws/aws-sdk-go-v2/credentials v1.13.18 h1:EQMdtHwz0ILTW1hoP+EwuWhwCG1hD6l3+RWFQABET4c= -github.com/aws/aws-sdk-go-v2/credentials v1.13.18/go.mod h1:vnwlwjIe+3XJPBYKu1et30ZPABG3VaXJYr8ryohpIyM= -github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.1 h1:gt57MN3liKiyGopcqgNzJb2+d9MJaKT/q1OksHNXVE4= -github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.1/go.mod h1:lfUx8puBRdM5lVVMQlwt2v+ofiG/X6Ms+dy0UkG/kXw= -github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.31 h1:sJLYcS+eZn5EeNINGHSCRAwUJMFVqklwkH36Vbyai7M= -github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.31/go.mod h1:QT0BqUvX1Bh2ABdTGnjqEjvjzrCfIniM9Sc8zn9Yndo= -github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.25 h1:1mnRASEKnkqsntcxHaysxwgVoUUp5dkiB+l3llKnqyg= -github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.25/go.mod h1:zBHOPwhBc3FlQjQJE/D3IfPWiWaQmT06Vq9aNukDo0k= -github.com/aws/aws-sdk-go-v2/internal/ini v1.3.32 h1:p5luUImdIqywn6JpQsW3tq5GNOxKmOnEpybzPx+d1lk= -github.com/aws/aws-sdk-go-v2/internal/ini v1.3.32/go.mod h1:XGhIBZDEgfqmFIugclZ6FU7v75nHhBDtzuB4xB/tEi4= -github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.25 h1:5LHn8JQ0qvjD9L9JhMtylnkcw7j05GDZqM9Oin6hpr0= -github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.25/go.mod h1:/95IA+0lMnzW6XzqYJRpjjsAbKEORVeO0anQqjd2CNU= -github.com/aws/aws-sdk-go-v2/service/sso v1.12.6 h1:5V7DWLBd7wTELVz5bPpwzYy/sikk0gsgZfj40X+l5OI= -github.com/aws/aws-sdk-go-v2/service/sso v1.12.6/go.mod h1:Y1VOmit/Fn6Tz1uFAeCO6Q7M2fmfXSCLeL5INVYsLuY= -github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.6 h1:B8cauxOH1W1v7rd8RdI/MWnoR4Ze0wIHWrb90qczxj4= -github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.6/go.mod h1:Lh/bc9XUf8CfOY6Jp5aIkQtN+j1mc+nExc+KXj9jx2s= -github.com/aws/aws-sdk-go-v2/service/sts v1.18.7 h1:bWNgNdRko2x6gqa0blfATqAZKZokPIeM1vfmQt2pnvM= -github.com/aws/aws-sdk-go-v2/service/sts v1.18.7/go.mod h1:JuTnSoeePXmMVe9G8NcjjwgOKEfZ4cOjMuT2IBT/2eI= +github.com/aws/aws-sdk-go-v2 v1.17.8 h1:GMupCNNI7FARX27L7GjCJM8NgivWbRgpjNI/hOQjFS8= +github.com/aws/aws-sdk-go-v2 v1.17.8/go.mod h1:uzbQtefpm44goOPmdKyAlXSNcwlRgF3ePWVW6EtJvvw= +github.com/aws/aws-sdk-go-v2/config v1.18.21 h1:ENTXWKwE8b9YXgQCsruGLhvA9bhg+RqAsL9XEMEsa2c= +github.com/aws/aws-sdk-go-v2/config v1.18.21/go.mod h1:+jPQiVPz1diRnjj6VGqWcLK6EzNmQ42l7J3OqGTLsSY= +github.com/aws/aws-sdk-go-v2/credentials v1.13.20 h1:oZCEFcrMppP/CNiS8myzv9JgOzq2s0d3v3MXYil/mxQ= +github.com/aws/aws-sdk-go-v2/credentials v1.13.20/go.mod h1:xtZnXErtbZ8YGXC3+8WfajpMBn5Ga/3ojZdxHq6iI8o= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.2 h1:jOzQAesnBFDmz93feqKnsTHsXrlwWORNZMFHMV+WLFU= +github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.2/go.mod h1:cDh1p6XkSGSwSRIArWRc6+UqAQ7x4alQ0QfpVR6f+co= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.32 h1:dpbVNUjczQ8Ae3QKHbpHBpfvaVkRdesxpTOe9pTouhU= +github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.32/go.mod h1:RudqOgadTWdcS3t/erPQo24pcVEoYyqj/kKW5Vya21I= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.26 h1:QH2kOS3Ht7x+u0gHCh06CXL/h6G8LQJFpZfFBYBNboo= +github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.26/go.mod h1:vq86l7956VgFr0/FWQ2BWnK07QC3WYsepKzy33qqY5U= +github.com/aws/aws-sdk-go-v2/internal/ini v1.3.33 h1:HbH1VjUgrCdLJ+4lnnuLI4iVNRvBbBELGaJ5f69ClA8= +github.com/aws/aws-sdk-go-v2/internal/ini v1.3.33/go.mod h1:zG2FcwjQarWaqXSCGpgcr3RSjZ6dHGguZSppUL0XR7Q= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.26 h1:uUt4XctZLhl9wBE1L8lobU3bVN8SNUP7T+olb0bWBO4= +github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.26/go.mod h1:Bd4C/4PkVGubtNe5iMXu5BNnaBi/9t/UsFspPt4ram8= +github.com/aws/aws-sdk-go-v2/service/sso v1.12.8 h1:5cb3D6xb006bPTqEfCNaEA6PPEfBXxxy4NNeX/44kGk= +github.com/aws/aws-sdk-go-v2/service/sso v1.12.8/go.mod h1:GNIveDnP+aE3jujyUSH5aZ/rktsTM5EvtKnCqBZawdw= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.8 h1:NZaj0ngZMzsubWZbrEFSB4rgSQRbFq38Sd6KBxHuOIU= +github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.8/go.mod h1:44qFP1g7pfd+U+sQHLPalAPKnyfTZjJsYR4xIwsJy5o= +github.com/aws/aws-sdk-go-v2/service/sts v1.18.9 h1:Qf1aWwnsNkyAoqDqmdM3nHwN78XQjec27LjM6b9vyfI= +github.com/aws/aws-sdk-go-v2/service/sts v1.18.9/go.mod h1:yyW88BEPXA2fGFyI2KCcZC3dNpiT0CZAHaF+i656/tQ= github.com/aws/smithy-go v1.13.5 h1:hgz0X/DX0dGqTYpGALqXJoRKRj5oQ7150i5FdTePzO8= github.com/aws/smithy-go v1.13.5/go.mod h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=