Kibana stats integration test failling on version key not found.

[ph]

found in https://travis-ci.org/elastic/beats/jobs/506466210

ok  	github.com/elastic/beats/metricbeat/module/kvm/dommemstat	0.007s	coverage: 48.1% of statements
command [go test -tags integration -cover -coverprofile /tmp/gotestcover-932530723 github.com/elastic/beats/metricbeat/module/kibana/stats]: exit status 1
Killing metricbeat43bb03fcf1fb8ccdc91622c74e2e2b37303cbde9_etcd_1 ... 
	stats_integration_test.go:60: 
			Error Trace:	stats_integration_test.go:60
			Error:      	Should be empty, but was [HTTP error 500 in stats: 500 Internal Server Error]
			Test:       	TestFetch
	stats_integration_test.go:61: 
			Error Trace:	stats_integration_test.go:61
			Error:      	Should NOT be empty, but was []
			Test:       	TestFetch
--- FAIL: TestData (1.03s)
	stats_integration_test.go:76: getting kibana version key not found
FAIL
coverage: 13.3% of statements
FAIL	github.com/elastic/beats/metricbeat/module/kibana/stats	149.446s

[elasticmachine]

Pinging @elastic/stack-monitoring

[ycombinator]

I've run this test locally about 20 times so far and no failures. I'd like to see from the next weekly flaky tests report if this is really a flaky test or a ephemeral failure.

[ph]

Since this is linked to a container, it's possible that the container is not completely ready in some cases. I've had similar issues with CM, worked successfully on a local (fast machine) but failed from time to time on Travis because of the slowness of the host.

[ycombinator]

Good point, @ph. Any tips how you repro'd/resolved the issue in the CM cases?

[ph]

I was a bit more strict on the health check strategies in my docker compose file and added a grace period to start with start_period.

@ycombinator What if you print the return of the API when there is an error? Maybe there is an error message when this happens?

[ycombinator]

Thanks @ph, I'll start by printing out the API response upon error. Let's see what we're dealing with and then take it from there.

[odacremolbap]

I'm facing what seems to be this issue consistently at #11759
@ycombinator @ph

Successfully built 658590a001b9
Successfully tagged metricbeateea8927e21d7387b78d71b7cdd072d24b8fb1246_kibana:latest
Creating metricbeateea8927e21d7387b78d71b7cdd072d24b8fb1246_kibana_1 ... 
    stats_integration_test.go:60: 
            Error Trace:    stats_integration_test.go:60
            Error:          Should be empty, but was [error making http request: Get http://kibana:5601/api/stats?extended=true: net/http: request canceled (Client.Timeout exceeded while awaiting headers)]
            Test:           TestFetch
    stats_integration_test.go:61: 
            Error Trace:    stats_integration_test.go:61
            Error:          Should NOT be empty, but was []
            Test:           TestFetch

[ycombinator]

Sorry about that @odacremolbap. I'm skipping these tests for now while we debug them: #11807.