Skip to content

SinceDB should support file offset and not only the keys #54

Open
@ph

Description

@ph

The current sincedb implementation of this plugin only relies on the object key and doesn't use the file offset at all, so when we stop logstash in a middle of reading a file we don't have the choice to read the file back at the beginning causing duplicates in the log stream.

We should investigate if we could use the filewatch plugins to actually do the file reading and the S3 input could act as a downloading agent.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions