Can't put document into AWS ES service.

[mthenw]

Which version of Elastic are you using?

elastic.v2 (for Elasticsearch 1.x)

Please describe the expected behavior

Successful document put into the index.

Please describe the actual behavior

Error is returned:

elastic: Error 403 (Forbidden)

Any steps to reproduce the behavior?

Setup:

    creds := credentials.NewEnvCredentials()
    signer := v4.NewSigner(creds)
    awsClient, err := aws_signing_client.New(signer, nil, "es", "us-west-2")
    if err != nil {
        return nil, err
    }

    return elastic.NewClient(
        elastic.SetURL(...),
        elastic.SetScheme("https"),
        elastic.SetHttpClient(awsClient),
        elastic.SetSniff(false),
    )

Put:

    _, err = e.Client.Index().Index(indexName).Type(indexType).
        Id(doc.ID).
        BodyJson(doc).
        Do()

Not sure if this is elastic or aws_signing_client issue.

[olivere]

@sha1sum Hi Anthony, do you have the time to reproduce this issue?

[olivere]

#301 and #303 use a different method for signing. If you're in a hurry, you could try that.

[mthenw]

I've solved that by creating http.Transport based on https://github.com/smartystreets/go-aws-auth

type AWSSigningTransport struct {
    HTTPClient  *http.Client
    Credentials awsauth.Credentials
}

// RoundTrip implementation
func (a AWSSigningTransport) RoundTrip(req *http.Request) (*http.Response, error) {
    return a.HTTPClient.Do(awsauth.Sign4(req, a.Credentials))
}

Usage

    signingTransport := AWSSigningTransport{
        Credentials: awsauth.Credentials{
            AccessKeyID:     os.Getenv("AWS_ACCESS_KEY"),
            SecretAccessKey: os.Getenv("AWS_SECRET_KEY"),
        },
        HTTPClient: http.DefaultClient,
    }
    signingClient := &http.Client{Transport: http.RoundTripper(signingTransport)}

    return elastic.NewClient(
        elastic.SetURL(...),
        elastic.SetScheme("https"),
        elastic.SetHttpClient(signingClient),
        elastic.SetSniff(false),
    )

[mthenw]

@olivere I found another bug. Looks like elastic creates wrong URL path (https://github.com/olivere/elastic/blob/release-branch.v2/search.go#L311-L344).

Requests go to URL//_search?search_type=count (double slash) instead of URL/_search?search_type=count. So for all search queries request signing doesn't work because of signature mismatch.

[olivere]

Yep. That's a bug. Thanks. Workaround is to use _all as an index name. Will hopefully find some time on the weekend.

[olivere]

The issue with the wrong URL being generated when indices and types are missing should be fixed in the next release later today.

I'm closing this issue for now as the problem with signing seems to be related to aws_signing_client#1, not elastic.

Feel free to re-open if this doesn't solve your problem. Thanks for your support.

[AlmogBaku]

I'm still having the same issue

[olivere]

@AlmogBaku If you've found a problem, don't be afraid to file a new issue. Put in a back-reference to this issue if you like. This issue actually talks about two bugs, and I'm not sure which one you're refering to.

[AlmogBaku]

actually, it was a bug of mine.. the included illegal chars which probably caused the failure to sign it. sorry for the trouble

[Single430]

This is the problem I encountered， help me!

no Elasticsearch node available
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x636df6]

[olivere]

@Single430 Can you post the code please? The 2nd line is because you probably didn't do any error checks and tried to use the *Client although it wasn't property initialized and is nil.

[Single430]

@olivere Ｏk , The first time use, there are mistakes please point out, thanks

type AWSSigningTransport struct {
    HTTPClient  *http.Client
    Credentials awsauth.Credentials
}

func (a AWSSigningTransport) RoundTrip(req *http.Request) (*http.Response, error) {
    return a.HTTPClient.Do(awsauth.Sign4(req, a.Credentials))
}

func EsSearch() {
    fmt.Println("search...")
    signingTransport := AWSSigningTransport{
        Credentials: awsauth.Credentials{
            AccessKeyID:     "xxxxxxxx",
            SecretAccessKey: "xxxxxxxxxxxxx",
        },
        HTTPClient: http.DefaultClient,
    }
    httpClient := &http.Client{
        Transport: http.RoundTripper(signingTransport),
    }
    client, err := elastic.NewClient(
        elastic.SetURL("https://search-xxxxxxx-xxxxxxx.cn-northwest-1.es.amazonaws.com.cn"),
        //elastic.SetScheme("https"),
        elastic.SetHealthcheckInterval(10*time.Second),
        elastic.SetHttpClient(httpClient),
        elastic.SetSniff(false),
        )
    if err != nil {
        fmt.Println(err)
    }
    
    termQuery := elastic.NewRangeQuery("published_time").Gte("2018-03-20T12:00:00").Lte("2018-03-21T12:00:00")
    searchResult, err := client.Search().
        Index("index").Type("type").
        Query(termQuery).
        Sort("published_time", true).
        From(0).Size(10).
        Pretty(true).
        Do()
    if err != nil {
        panic(err)
    }
    
    fmt.Printf("Query took %d milliseconds\n", searchResult.TookInMillis)
    fmt.Println(searchResult.Hits.TotalHits)
    
    fmt.Println("Hello World!")
}

func main() {
    EsSearch()
}