Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate forward plugin to v0.14 api #1306

Merged
merged 10 commits into from
Nov 9, 2016

Conversation

tagomoris
Copy link
Member

@tagomoris tagomoris commented Nov 7, 2016

This change is to migrate in/out_forward to v0.14 APIs, except for socket/server plugin helpers, to keep commits few and diff small.

@tagomoris tagomoris added the v0.14 label Nov 7, 2016
@tagomoris tagomoris self-assigned this Nov 7, 2016
@tagomoris tagomoris force-pushed the migrate-forward-plugin-to-v0.14-api branch from b2de9a0 to 4d9e1a7 Compare November 7, 2016 09:47
@tagomoris tagomoris changed the title [WIP] Migrate forward plugin to v0.14 api Migrate forward plugin to v0.14 api Nov 8, 2016
@tagomoris
Copy link
Member Author

Pull requests to add socket/server plugin helpers will follow this change.
@repeatedly Could you review this?

@tagomoris tagomoris assigned repeatedly and unassigned tagomoris Nov 8, 2016
@repeatedly
Copy link
Member

Okay. I will review it later 👍

@tagomoris
Copy link
Member Author

@repeatedly ping?

@thread.join
@lsock.close

def stop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stop is called before shutdown, right?
What happens when close socket before shutdown/close event loop?
Is it safe?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, it's mistake and it should be done in #close. I'll fix it.

require 'fluent/input'
require 'fluent/plugin/socket_util'
require 'fcntl'
require 'cool.io'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool.io is required by event_loop helper.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah wait. This commit doesn't include socket helpers. Ignore this comment.

class Error < StandardError; end
class ResponseError < Error; end
class ConnectionClosedError < Error; end
class ACKTimeoutError < Error; end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In v0.12, ForwardOutputACKTimeoutError is subclass of ForwardOutputResponseError.
Change from ResponseError to Error is intented?

Copy link
Member Author

@tagomoris tagomoris Nov 9, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's intended. There are no actual benefits to make these errors with sub-super classes.
And, in fact, there are no responses under ACK timeout situation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay

["tag2", time, {"a"=>2}],
]

d.run(expect_records: records.length, timeout: 5) do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why change timeout to 5?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for stability of CI tests.

d.run do
entries = records.map { |tag, _time, record| [tag, _time, record] }
d.run(shutdown: false, expect_records: 2, timeout: 10) do
entries = []
# These entries are skipped
entries << ['tag1', true, {'a' => 3}] << ['tag2', time, 'invalid record']
Copy link
Member

@repeatedly repeatedly Nov 9, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use 2 lines instead of entries = [[...], [...]]?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To reduce the number of lines of diff.

attr_reader :responses, :exceptions

def write(chunk)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this definition needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Output#implement? checks the features of that plugin by checking instance methods implemented in just that class, without methods of superclasses.
This test code defines a one-time class inherits ForwardOutput. If this method is not defined, Output#implement? returns false for check of buffered output feature.

It's better to be fixed in future, but currently, this code works fine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I understood the situation.

@repeatedly
Copy link
Member

Commented. Others looks good.

@tagomoris
Copy link
Member Author

I pushed a commit to close sockets in #close, and will merge this after CI green.

@tagomoris tagomoris merged commit a3a8444 into master Nov 9, 2016
@tagomoris tagomoris deleted the migrate-forward-plugin-to-v0.14-api branch November 9, 2016 06:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants