Shutdown all record processors in an orderly fashion when we shutdown the JVM

[xujiaxj]

We often use JVM shutdown hook to make sure the processing pipeline is flushed in an orderly server shutdown.

For a record processor, what needs to be done in such case is pretty much the same as handling the "TERMINATE" command, i.e., finish the processing and checkpoint. For the current KCL design, however, to implement the JVM shutdown hook needs to manually keep track of all record processors and issue "TERMINATE" ourselves.

Ideally the Worker class's shutdown() method should take care of this, so we only need to call worker.shutdown(), but right now, it doesn't do that, and what it does seems to be no more than excluding itself from lease coordination.

[az3]

I agree, this functionality is needed.

See line 325 on Worker.java

amazon-kinesis-client/src/main/java/com/amazonaws/services/kinesis/clientlibrary/lib/worker/Worker.java

Line 325 in c6e393c

while (!shutdown) {

For example, if you catch TERM signal and stop your worker thread; worker shuts itself down and the lease coordinator. So the current running RecordProcessor classes will fail upon calling recordProcessorCheckpointer.checkpoint() method due to missing lease.

INFO [KinesisClientLibLeaseCoordinator] Worker az3-pc could not update checkpoint for shard shardId-000000000048 because it does not hold the lease
com.amazonaws.services.kinesis.clientlibrary.exceptions.ShutdownException: Can't update checkpoint - instance doesn't hold the lease for this shard

After exiting the while loop, and just before calling "leaseCoordinator.stop()" worker thread should tell all running threads, that it is exiting.

[pfifer]

I'm currently working to add support for orderly shutdown right now. I've put some comments in #79. I don't currently have a release date yet, but I do hope to have it soon.

[pfifer]

Graceful shutdown was added in Release 1.7.1.

This doesn't add shutdown hooks for the JVM so you would need to add those yourself.

Feel free to reopen if you have any other questions.