Can't connect to AWS Elasticsearch Service

[pierrefar]

Hi

We have an AWS ES service that allows only a specified set of IP addresses to access the domain - no usernames. This works as expected, but Mirage fails with an authentication error using the Chrome extension running on an IP address that works.

[farhan687]

Hi @pierrefar Can you show me the 'url' format?

[pierrefar]

@farhan687 It's like this:

https://long_random_string_identifying_ES_domain.us-west-2.es.amazonaws.com

Of course "us-west-2" will depend on the AWS region you're using. Not sure if HTTP works as we always use HTTPS.

[farhan687]

hi @pierrefar Can you try it again? I have updated the mirage for gh-pages and elasticsearch plugin.

[pierrefar]

Hi @farhan687 Not working unfortunately. I tried different indices, and different combinations of the URL specifying HTTP/HTTPS, specifying different ports (:80, :443, :9200). Mirage refuses to do anything unless it sees :9200, and then it fails to authenticate after a few seconds of trying.

[Garrett-R]

A more simple example of what I believe to be the same bug is not being able to connect to http://localhost:8080/ even though the server is running fine:

~ $ curl http://localhost:8080
{
  "name" : "Hood",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "2.4.0",
    "build_hash" : "ce9f0c7394dee074091dd1bc4e9469251181fc55",
    "build_timestamp" : "2016-08-29T09:14:17Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.2"
  },
  "tagline" : "You Know, for Search"
}

[farhan687]

Sorry for inconvenience, we are trying to solve this issue which is specific to mirage chrome-extension.
For now you can use mirage elasticsearch plugin which is working fine.
I will inform you once we resolve this issue.

[farhan687]

Hey @Garrett-R @pierrefar sorry for the delay, it took some time to just figure out the issue and now we fixed the issue.
The issue regarding chrome-extension is fixed and updated mirage on chrome store.
Can you try mirage with mirage 0.1.5 version?

[siddharthlatest]

@Garrett-R do you have an index name selected when you try to "connect"?

[Garrett-R]

@siddharthlatest, whoops, thanks, I forgot that piece. (I've delete my comment)

[Garrett-R]

OK, so, trying again on the hosted site, I'm still getting an error. Here's what it looks like before I hit Connect:

Note that the second tab is showing that localhost:8080 is indeed serving up properly.

I hit connect and get this error:

It looks like your app name, username, password combination doesn't match.
Check your url and appname and then connect it again.

[farhan687]

@Garrett-R Looks like CORS issue, Can you check in console what does it shows, when you are trying to connect.

Basic workaround for CORS issue with elasticsearch:
Apply following changes in elasticsearch:

http.cors.allow-origin: "*"
http.cors.enabled: true
http.cors.allow-headers : "X-Requested-With, X-Auth-Token, Content-Type, Content-Length, Authorization"
http.cors.allow-methods: "OPTIONS, HEAD, GET, POST, PUT, DELETE"
http.cors.allow-credentials: true

For more information:
elastic/elasticsearch#9063

[Garrett-R]

@farhan687, thank you. Adding those lines to elasticsearch.yml did indeed solve the problem.

[pierrefar]

@farhan687 Still not working with AWS unfortunately.

What I did: Deleted the extension from Chrome and installed it fresh. Gave it the correct AWS ES endpoint and a correct index name. Failed with the same authentication error.

[farhan687]

@pierrefar Can you take screenshot of error log?

[pierrefar]

@farhan687 Attached. It's the same text as @Garrett-R copied earlier.

[farhan687]

@pierrefar Can you share elasticsearch version info?

[pierrefar]

Hi @farhan687 . It's ES 1.5.2, the standard one hosted on AWS. And as of a few weeks ago they have a newer ES version.

They have a free usage tier if you want to test with one directly.

[siddharthlatest]

@pierrefar Mirage only works for 2.x and above versions. I would recommend you trying their newer version.

Elasticsearch has an issue with CORS on v2.3.0 and v2.3.1, see reference - elastic/elasticsearch#17483.

Your best bet is using v2.4.0 and above.

[andresilveirah]

Hi guys, sorry for reviving this issue.
I'm having the same problem to use Mirage with an unprotected instance of ES running on AWS.
The funny part is that I can connect without problems using Dejavu.

Any ideas?

[mschnee]

The Authorization header is something that AWS-hosted Elasticsearch EC2 container itself is complaining about. However, the fact that mirage is injecting it is a huge problem because it does not appear to be coming from any kind of actual Oauth flow.

Four things:
First- if we use AWS-hosted Elasticsearch, we can't change CORS.
Second- AWS hosted Elasticsearch service works fine.
Third- Unless we provide an "Authorization" header, which is not supported (or needed).
Fourth- ...which Mirage provides, and cannot be removed.

curl https://vpc-id-somehash.us-east-1.es.amazonaws.com/someindex/_mapping/
{"someindex": {"mappings":{...}}}

curl https://vpc-id-somehash.us-east-1.es.amazonaws.com/someindex/_mapping/ -H "Authorization: lolwhat"
{"message":"Authorization header requires 'Credential' parameter. Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization header requires existence of either a 'X-Amz-Date' or a 'Date' header. Authorization=lolwhat"}

So question for @siddharthlatest

• What is this Authorization header? Where is it coming from? I'm in a fresh, Private Mode browser, connected through a VPN. I have no cookies, no sessions, no localstorage. What are you trying to authorize me against without my say-so?
• Why can't I turn this off? It's unnecessary.
• Is this something specific for appbase?

It should be noted that dejavu works fine and does not forcefully inject this magic Authorization header.