KCL not resilient to network issues

[witoff]

We frequently see the below error that takes down our nodes: The kcl should be more resilient on a box that's meant to process a firehose. What's worse, is that our error handling & instrumentation does not pick up java errors -- we'd really like to support a native ruby implementation.

at java.lang.Thread.run(Thread.java:745)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.MetricsCollectingTaskDecorator.call(MetricsCollectingTaskDecorator.java:23)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.MetricsCollectingTaskDecorator.call(MetricsCollectingTaskDecorator.java:48)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.ProcessTask.call(ProcessTask.java:96)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.ProcessTask.getRecords(ProcessTask.java:186)
at com.amazonaws.services.kinesis.clientlibrary.lib.worker.KinesisDataFetcher.getRecords(KinesisDataFetcher.java:69)
at com.amazonaws.services.kinesis.clientlibrary.proxies.MetricsCollectingKinesisProxyDecorator.get(MetricsCollectingKinesisProxyDecorator.java:72)
at com.amazonaws.services.kinesis.clientlibrary.proxies.KinesisProxy.get(KinesisProxy.java:147)
at com.amazonaws.services.kinesis.AmazonKinesisClient.getRecords(AmazonKinesisClient.java:592)
at com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:2130)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:245)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:417)
at com.amazonaws.http.AmazonHttpClient.handleResponse(AmazonHttpClient.java:730)
at com.amazonaws.http.JsonResponseHandler.handle(JsonResponseHandler.java:41)
at com.amazonaws.http.JsonResponseHandler.handle(JsonResponseHandler.java:95)
at com.fasterxml.jackson.core.JsonFactory.createParser(JsonFactory.java:671)
at com.fasterxml.jackson.core.JsonFactory.createJsonParser(JsonFactory.java:831)
at com.fasterxml.jackson.core.JsonFactory._createParser(JsonFactory.java:1182)
at com.fasterxml.jackson.core.JsonFactory._createJsonParser(JsonFactory.java:1191)
at com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.constructParser(ByteSourceJsonBootstrapper.java:226)
at com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.detectEncoding(ByteSourceJsonBootstrapper.java:129)
at com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.ensureLoaded(ByteSourceJsonBootstrapper.java:505)
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:138)
at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:182)
at org.apache.http.impl.io.AbstractSessionInputBuffer.read(AbstractSessionInputBuffer.java:204)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:918)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:961)
at sun.security.ssl.InputRecord.read(InputRecord.java:532)
at sun.security.ssl.InputRecord.readV3Record(InputRecord.java:593)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketTimeoutException: Read timed out
INFO: Unable to execute HTTP request: Read timed out

[Cory-Bradshaw]

Due to the lack of conversation here, I will close this issue for now. If you or other customers experiance this issue still, please provide details on your setup that is leading to this occurring frequently, and reopen the issue.

For supporting a Native Ruby KCL please leave comments on #27