Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor shutdown sequence to use plugin.stop #53

Closed
jsvd opened this issue Sep 11, 2015 · 1 comment
Closed

refactor shutdown sequence to use plugin.stop #53

jsvd opened this issue Sep 11, 2015 · 1 comment
Assignees
@ph
Labels
shutdown semantics

Comments

@jsvd
Copy link
Member

jsvd commented Sep 11, 2015

Change shutdown sequence to be triggered by plugin.stop instead of ShutdownSignal exception.
Also remove any calls to: shutdown, finished, finished?, running? or terminating?
This depends on elastic/logstash#3895
@jsvd jsvd mentioned this issue Sep 11, 2015
Closed
43 tasks
@jsvd jsvd assigned ph Sep 11, 2015
@ph
Copy link
Contributor

ph commented Sep 14, 2015

depends on jordansissel/ruby-stud#27

ph added a commit to ph/logstash-input-s3 that referenced this issue Sep 16, 2015
@ph
Refactor the shutdown sequence of this plugin to follow the new semantic
8e4fdb0 
This plugin is now interruptable, this plugin require a bit more care to
shutdown it down for a few reasons:

1. It can be stuck streaming file to disk from a S3 bucket.
2. It can be stuck reading a large file

For case 1, we cancel the streaming and delete the incomplete temp file
and not update the sincedb.

For case 2, we will cancel reading the file and not update the sincedb,
this will result in duplicates in the logging stream which is fine for now since the S3 plugin
doesn't support saving offset yet. see logstash-plugins#54

This PR also add a fix to non deterministic test run in a local
environment by specifying a new sincedb for each run.

Fixes: logstash-plugins#53 logstash-plugins#42
@ph ph mentioned this issue Sep 16, 2015
Closed
ph added a commit to ph/logstash-input-s3 that referenced this issue Sep 16, 2015
@ph
Refactor the shutdown sequence of this plugin to follow the new semantic
7c64775 
This plugin is now interruptable, this plugin require a bit more care to
shutdown it down for a few reasons:

1. It can be stuck streaming file to disk from a S3 bucket.
2. It can be stuck reading a large file

For case 1, we cancel the streaming and delete the incomplete temp file
and not update the sincedb.

For case 2, we will cancel reading the file and not update the sincedb,
this will result in duplicates in the logging stream which is fine for now since the S3 plugin
doesn't support saving offset yet. see logstash-plugins#54

This PR also add a fix to non deterministic test run in a local
environment by specifying a new sincedb for each run.

Fixes: logstash-plugins#53 logstash-plugins#42
ph added a commit to ph/logstash-input-s3 that referenced this issue Sep 21, 2015
@ph
Refactor the shutdown sequence of this plugin to follow the new semantic
ebe47cc 
This plugin is now interruptable, this plugin require a bit more care to
shutdown it down for a few reasons:

1. It can be stuck streaming file to disk from a S3 bucket.
2. It can be stuck reading a large file

For case 1, we cancel the streaming and delete the incomplete temp file
and not update the sincedb.

For case 2, we will cancel reading the file and not update the sincedb,
this will result in duplicates in the logging stream which is fine for now since the S3 plugin
doesn't support saving offset yet. see logstash-plugins#54

This PR also add a fix to non deterministic test run in a local
environment by specifying a new sincedb for each run.

Fixes: logstash-plugins#53 logstash-plugins#42
ph added a commit to ph/logstash-input-s3 that referenced this issue Sep 22, 2015
@ph
Refactor the shutdown sequence of this plugin to follow the new semantic
dbf7745 
This plugin is now interruptable, this plugin require a bit more care to
shutdown it down for a few reasons:

1. It can be stuck streaming file to disk from a S3 bucket.
2. It can be stuck reading a large file

For case 1, we cancel the streaming and delete the incomplete temp file
and not update the sincedb.

For case 2, we will cancel reading the file and not update the sincedb,
this will result in duplicates in the logging stream which is fine for now since the S3 plugin
doesn't support saving offset yet. see logstash-plugins#54

This PR also add a fix to non deterministic test run in a local
environment by specifying a new sincedb for each run.

Fixes: logstash-plugins#53 logstash-plugins#42
ph added a commit that referenced this issue Sep 22, 2015
@ph @jordansissel
Refactor the shutdown sequence of this plugin to follow the new semantic
221bb96 
This plugin is now interruptable, this plugin require a bit more care to
shutdown it down for a few reasons:

1. It can be stuck streaming file to disk from a S3 bucket.
2. It can be stuck reading a large file

For case 1, we cancel the streaming and delete the incomplete temp file
and not update the sincedb.

For case 2, we will cancel reading the file and not update the sincedb,
this will result in duplicates in the logging stream which is fine for now since the S3 plugin
doesn't support saving offset yet. see #54

This PR also add a fix to non deterministic test run in a local
environment by specifying a new sincedb for each run.

Fixes: #53 #42

Fixes #55
@ph ph closed this as completed Sep 22, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ph ph

Labels
shutdown semantics
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Refactor the shutdown sequence of this plugin to follow the new semantic ph/logstash-input-s3
2 participants
@ph @jsvd