Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

if router enable http/2, Unable to open kibana #1465

Closed
klzsysy opened this issue Nov 27, 2018 · 12 comments
Closed

if router enable http/2, Unable to open kibana #1465

klzsysy opened this issue Nov 27, 2018 · 12 comments
Labels
lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed.

Comments

@klzsysy
Copy link

klzsysy commented Nov 27, 2018

for openshift 3.11

if enable router http2:

oc set env dc/router ROUTER_ENABLE_HTTP2=true

image

image: docker.io/openshift/origin-logging-kibana5:v3.11
@richm
Copy link
Contributor

richm commented Nov 27, 2018

can you provide the elasticsearch and kibana pod logs?

@josefkarasek
Copy link

@richm the re-encrypt route might be the issue here.
If it were passthrough, the router just works with ssl and doesn't care about whatever is wrapped inside.
But because it unwraps the ssl layer, it is exposed to the http later underneath.

If this is really the case, I think that it's up for discussion if we want to support such router configuration.

@klzsysy
Copy link
Author

klzsysy commented Nov 28, 2018

kibana log:

{"type":"response","@timestamp":"2018-11-28T02:22:28Z","tags":[],"pid":223,"method":"head","statusCode":200,"req":{"url":"/","method":"head","headers":{"user-agent":"curl/7.29.0","host":"localhost:5601","accept":"*/*"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1"},"res":{"statusCode":200,"responseTime":2,"contentLength":9},"message":"HEAD / 200 2ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:30Z","tags":[],"pid":223,"method":"get","statusCode":200,"req":{"url":"/app/kibana","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","cache-control":"max-age=0","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","referer":"https://kibana.localprod.forc.work/app/kibana","upgrade-insecure-requests":"1","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":200,"responseTime":424,"contentLength":9},"message":"GET  /app/kibana 200 424ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:31Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/ui/fonts/open_sans/open_sans_v13_latin_300.woff2","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"*/*","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Tue, 30 Oct 2018 19:17:29 GMT","if-none-match":"\"2da4d6d1b1bef32c05719680145b61613b3829b4\"","origin":"https://kibana.localprod.forc.work","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":3,"contentLength":9},"message":"GET  /ui/fonts/open_sans/open_sans_v13_latin_300.woff2 304 3ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:32Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/bundles/kibana.style.css?v=15661","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"text/css,*/*;q=0.1","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Mon, 19 Nov 2018 17:32:04 GMT","if-none-match":"\"238977513b62015b837390a3256e0be5bcd3cecd-gzip\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":5,"contentLength":9},"message":"GET  /bundles/kibana.style.css?v=15661 304 5ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:32Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/bundles/commons.style.css?v=15661","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"text/css,*/*;q=0.1","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Mon, 19 Nov 2018 17:32:04 GMT","if-none-match":"\"3d833de3ea9ab38f9ccc539aafd4d489b3f7f81b-gzip\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":6,"contentLength":9},"message":"GET  /bundles/commons.style.css?v=15661 304 6ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:32Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/bundles/commons.bundle.js?v=15661","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"*/*","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Mon, 19 Nov 2018 17:32:04 GMT","if-none-match":"\"36b3ef59c8c2c36ee91d4dcf5aacfcd297e6af4f-gzip\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":2,"contentLength":9},"message":"GET  /bundles/commons.bundle.js?v=15661 304 2ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:33Z","tags":[],"pid":223,"method":"get","statusCode":200,"req":{"url":"/ui/favicons/favicon-32x32.png","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","cache-control":"no-cache","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","pragma":"no-cache","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":200,"responseTime":6,"contentLength":9},"message":"GET  /ui/favicons/favicon-32x32.png 200 6ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:33Z","tags":[],"pid":223,"method":"head","statusCode":200,"req":{"url":"/","method":"head","headers":{"user-agent":"curl/7.29.0","host":"localhost:5601","accept":"*/*"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1"},"res":{"statusCode":200,"responseTime":3,"contentLength":9},"message":"HEAD / 200 3ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:34Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/ui/favicons/favicon-16x16.png","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Tue, 30 Oct 2018 19:17:29 GMT","if-none-match":"\"f33f077bfe13045136046c93b6180be0379386ff\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":18,"contentLength":9},"message":"GET  /ui/favicons/favicon-16x16.png 304 18ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:34Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/bundles/kibana.bundle.js?v=15661","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"*/*","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Mon, 19 Nov 2018 17:32:04 GMT","if-none-match":"\"848e7453075ed2635cebc9ed7e7eac941b253aea-gzip\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":3,"contentLength":9},"message":"GET  /bundles/kibana.bundle.js?v=15661 304 3ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:34Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/ui/favicons/favicon.ico","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Tue, 30 Oct 2018 19:17:29 GMT","if-none-match":"\"72df422636eb8c15a5f38607b794cb22f67f6dc5-gzip\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":30,"contentLength":9},"message":"GET  /ui/favicons/favicon.ico 304 30ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:35Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/bundles/45b47f3e9c7d74b80f5c6e0a3c513b23.woff","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"*/*","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Mon, 19 Nov 2018 17:32:04 GMT","if-none-match":"\"523f9403d934ba017bfe924cdc40f1bb0e5d97e2\"","origin":"https://kibana.localprod.forc.work","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661"},"res":{"statusCode":304,"responseTime":4,"contentLength":9},"message":"GET  /bundles/45b47f3e9c7d74b80f5c6e0a3c513b23.woff 304 4ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:35Z","tags":[],"pid":223,"method":"get","statusCode":200,"req":{"url":"/api/console/api_server?sense_version=%40%40SENSE_VERSION&apis=es_5_0","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"application/json, text/javascript, */*; q=0.01","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","kbn-version":"5.6.13","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny","x-requested-with":"XMLHttpRequest"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":200,"responseTime":7,"contentLength":9},"message":"GET  /api/console/api_server?sense_version=%40%40SENSE_VERSION&apis=es_5_0 200 7ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:35Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/plugins/kibana/assets/discover.svg","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Tue, 30 Oct 2018 19:17:29 GMT","if-none-match":"\"c4035451a8e776d0f0cd354a825ec432ad06884e-gzip\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":3,"contentLength":9},"message":"GET  /plugins/kibana/assets/discover.svg 304 3ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:35Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/plugins/kibana/assets/visualize.svg","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Tue, 30 Oct 2018 19:17:29 GMT","if-none-match":"\"4cc79a4d91bd0380d0c82a6b092f339d185670ef-gzip\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":3,"contentLength":9},"message":"GET  /plugins/kibana/assets/visualize.svg 304 3ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:35Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/plugins/kibana/assets/dashboard.svg","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Tue, 30 Oct 2018 19:17:29 GMT","if-none-match":"\"42c2161fa64691414784868afdd722444460763a-gzip\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":5,"contentLength":9},"message":"GET  /plugins/kibana/assets/dashboard.svg 304 5ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:35Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/plugins/timelion/icon.svg","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Tue, 30 Oct 2018 19:17:29 GMT","if-none-match":"\"cb793d5314d680b7d5ce130f0393a70b51989541-gzip\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":2,"contentLength":9},"message":"GET  /plugins/timelion/icon.svg 304 2ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:36Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/plugins/kibana/assets/wrench.svg","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Tue, 30 Oct 2018 19:17:29 GMT","if-none-match":"\"088a9a98c99e406dca2354af14f688ad84826b97-gzip\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":2,"contentLength":9},"message":"GET  /plugins/kibana/assets/wrench.svg 304 2ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:36Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/plugins/kibana/assets/settings.svg","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Tue, 30 Oct 2018 19:17:29 GMT","if-none-match":"\"4f859e27d4917026ff1590805887902b14ce79d5-gzip\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":1,"contentLength":9},"message":"GET  /plugins/kibana/assets/settings.svg 304 1ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:36Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/plugins/kibana/assets/play-circle.svg","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Tue, 30 Oct 2018 19:17:29 GMT","if-none-match":"\"2433ecf38258f7121c835670b6993600e7657717-gzip\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":1,"contentLength":9},"message":"GET  /plugins/kibana/assets/play-circle.svg 304 1ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:35Z","tags":[],"pid":223,"method":"get","statusCode":200,"req":{"url":"/api/saved_objects/?type=index-pattern&per_page=10000&page=1","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"application/json, text/plain, */*","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","kbn-version":"5.6.13","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":200,"responseTime":1186,"contentLength":9},"message":"GET  /api/saved_objects/?type=index-pattern&per_page=10000&page=1 200 1186ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:37Z","tags":[],"pid":223,"method":"get","statusCode":200,"req":{"url":"/api/saved_objects/?type=index-pattern&fields=title&per_page=10000&page=1","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"application/json, text/plain, */*","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","kbn-version":"5.6.13","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":200,"responseTime":909,"contentLength":9},"message":"GET  /api/saved_objects/?type=index-pattern&fields=title&per_page=10000&page=1 200 909ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:38Z","tags":[],"pid":223,"method":"head","statusCode":200,"req":{"url":"/","method":"head","headers":{"user-agent":"curl/7.29.0","host":"localhost:5601","accept":"*/*"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1"},"res":{"statusCode":200,"responseTime":2,"contentLength":9},"message":"HEAD / 200 2ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:38Z","tags":[],"pid":223,"method":"post","statusCode":200,"req":{"url":"/api/saved_objects/bulk_get","method":"post","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","content-length":"47","accept":"application/json, text/plain, */*","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","content-type":"application/json;charset=UTF-8","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","kbn-version":"5.6.13","origin":"https://kibana.localprod.forc.work","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":200,"responseTime":298,"contentLength":9},"message":"POST  /api/saved_objects/bulk_get 200 298ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:39Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/bundles/697574b47bcfdd2c45e3e63c7380dd67.woff","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"*/*","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Mon, 19 Nov 2018 17:32:04 GMT","if-none-match":"\"4590722b795938e0b6ff1b99701d1abe37aeabef\"","origin":"https://kibana.localprod.forc.work","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661"},"res":{"statusCode":304,"responseTime":3,"contentLength":9},"message":"GET  /bundles/697574b47bcfdd2c45e3e63c7380dd67.woff 304 3ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:39Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/bundles/97493d3f11c0a3bd5cbd959f5d19b699.woff2","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"*/*","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Mon, 19 Nov 2018 17:32:04 GMT","if-none-match":"\"1075231650f579955905bb2f6527148a8e2b4b16\"","origin":"https://kibana.localprod.forc.work","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661"},"res":{"statusCode":304,"responseTime":4,"contentLength":9},"message":"GET  /bundles/97493d3f11c0a3bd5cbd959f5d19b699.woff2 304 4ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:39Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/bundles/c0ae3861f54b68fa4e603c9877eff7a9.svg","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Mon, 19 Nov 2018 17:32:04 GMT","if-none-match":"\"61cd797076c727bb7fe0459f70c0bdaf7356dc94-gzip\"","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661"},"res":{"statusCode":304,"responseTime":4,"contentLength":9},"message":"GET  /bundles/c0ae3861f54b68fa4e603c9877eff7a9.svg 304 4ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:40Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/bundles/2e90d5152ce92858b62ba053c7b9d2cb.woff","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"*/*","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Mon, 19 Nov 2018 17:32:04 GMT","if-none-match":"\"8cf65f42a2a8c349ccd6ab63b6cbd17c96fd665c\"","origin":"https://kibana.localprod.forc.work","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661"},"res":{"statusCode":304,"responseTime":5,"contentLength":9},"message":"GET  /bundles/2e90d5152ce92858b62ba053c7b9d2cb.woff 304 5ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:40Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/bundles/79515ad0788973c533405f7012dfeccd.woff","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"*/*","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Mon, 19 Nov 2018 17:32:04 GMT","if-none-match":"\"5092881fad2caffdc6bf71bdab1ea547b73d3564\"","origin":"https://kibana.localprod.forc.work","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661"},"res":{"statusCode":304,"responseTime":2,"contentLength":9},"message":"GET  /bundles/79515ad0788973c533405f7012dfeccd.woff 304 2ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:40Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/bundles/c7a170b07cdeddc1f797b3ed47eb9991.ttf","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"*/*","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Mon, 19 Nov 2018 17:32:04 GMT","if-none-match":"\"1eb1dbbd7add61d80c6c2198238c445fdf6ab005\"","origin":"https://kibana.localprod.forc.work","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661"},"res":{"statusCode":304,"responseTime":1,"contentLength":9},"message":"GET  /bundles/c7a170b07cdeddc1f797b3ed47eb9991.ttf 304 1ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:40Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/bundles/448c34a56d699c29117adc64c43affeb.woff2","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"*/*","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Mon, 19 Nov 2018 17:32:04 GMT","if-none-match":"\"ca35b697d99cae4d1b60f2d60fcd37771987eb07\"","origin":"https://kibana.localprod.forc.work","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/bundles/commons.style.css?v=15661"},"res":{"statusCode":304,"responseTime":2,"contentLength":9},"message":"GET  /bundles/448c34a56d699c29117adc64c43affeb.woff2 304 2ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:40Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/ui/favicons/favicon-32x32.png","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Tue, 30 Oct 2018 19:17:29 GMT","if-none-match":"\"128bd421ef426ccdd9ea3293f79d25687b9832e7\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":3,"contentLength":9},"message":"GET  /ui/favicons/favicon-32x32.png 304 3ms - 9.0B"}
{"type":"response","@timestamp":"2018-11-28T02:22:40Z","tags":[],"pid":223,"method":"get","statusCode":304,"req":{"url":"/ui/favicons/favicon-16x16.png","method":"get","headers":{"host":"kibana.localprod.forc.work","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"zh-CN,zh;q=0.9,en-US;q=0.8,en;q=0.7","forwarded":"for=10.0.21.251;host=kibana.localprod.forc.work;proto=https;proto-version=h2","if-modified-since":"Tue, 30 Oct 2018 19:17:29 GMT","if-none-match":"\"f33f077bfe13045136046c93b6180be0379386ff\"","referer":"https://kibana.localprod.forc.work/app/kibana","x-forwarded-access-token":"vvNqQsv_nSjf3dShJgqOH_GZ4OOhrC0gsq8NoLq8U8s","x-forwarded-email":"Sonny@cluster.local","x-forwarded-for":"10.0.21.251 , 172.21.0.1","x-forwarded-host":"kibana.localprod.forc.work","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-proto-version":"h2","x-forwarded-user":"Sonny"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana.localprod.forc.work/app/kibana"},"res":{"statusCode":304,"responseTime":3,"contentLength":9},"message":"GET  /ui/favicons/favicon-16x16.png 304 3ms - 9.0B"}

es no access log

20181128102510

@richm

@richm
Copy link
Contributor

richm commented Nov 28, 2018

I don't see any obvious errors in the kibana log?
what about the kibana-proxy log? oc logs -c kibana-proxy $kibana-pod

@klzsysy
Copy link
Author

klzsysy commented Nov 28, 2018 

2018/11/27 07:21:44 provider.go:526: Performing OAuth discovery against https://172.30.0.1/.well-known/oauth-authorization-server 
2018/11/27 07:21:45 provider.go:572: 200 GET https://172.30.0.1/.well-known/oauth-authorization-server  {
  "issuer": "https://localprod.forc.work:8443 ",
  "authorization_endpoint": "https://localprod.forc.work:8443/oauth/authorize ",
  "token_endpoint": "https://localprod.forc.work:8443/oauth/token ",
  "scopes_supported": [
    "user:check-access",
    "user:full",
    "user:info",
    "user:list-projects",
    "user:list-scoped-projects"
  ],
  "response_types_supported": [
    "code",
    "token"
  ],
  "grant_types_supported": [
    "authorization_code",
    "implicit"
  ],
  "code_challenge_methods_supported": [
    "plain",
    "S256"
  ]
}
2018/11/27 07:21:45 oauthproxy.go:201: mapping path "/" => upstream "http://localhost:5601/ "
2018/11/27 07:21:45 oauthproxy.go:228: OAuthProxy configured for  Client ID: kibana-proxy
2018/11/27 07:21:45 oauthproxy.go:238: Cookie settings: name:_oauth_proxy secure(https):true httponly:true expiry:168h0m0s domain:<default> refresh:disabled
2018/11/27 07:21:45 http.go:56: HTTP: listening on 127.0.0.1:4180
2018/11/27 07:21:45 http.go:96: HTTPS: listening on [::]:3000
2018/11/28 02:22:12 reverseproxy.go:316: http: proxy error: context canceled
2018/11/28 02:22:40 reverseproxy.go:316: http: proxy error: context canceled

@jcantrill
Copy link
Contributor

Is this an openshift issue or a kibana issue? Seems there is an open issue upstream to support http2

elastic/kibana#7104

@Asgoret
Copy link

Asgoret commented Jun 14, 2019 •
edited
Loading

Same issue with images:

  • docker.io/openshift/origin-logging-kibana5:v3.11
  • docker.io/openshift/oauth-proxy:v1.1.0

@josefkarasek Hi! How you configure kibana as workaround?

TIP: We need H2 support for another application)

@Asgoret Asgoret mentioned this issue Jun 14, 2019
Open
@richm
Copy link
Contributor

richm commented Aug 30, 2019

@ewolinetz not sure if your recent proxy work is relevant

@openshift-bot
Copy link

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

@openshift-ci-robot openshift-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 16, 2020
@openshift-bot
Copy link

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

@openshift-ci-robot openshift-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Oct 16, 2020
@openshift-bot
Copy link

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

@openshift-ci-robot
Copy link

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@richm @openshift-bot @klzsysy @jcantrill @josefkarasek @openshift-ci-robot @Asgoret