Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Journalbeat: add index option to input #15071

Merged
merged 11 commits into from
Dec 13, 2019

Conversation

ycombinator
Copy link
Contributor

@ycombinator ycombinator commented Dec 11, 2019

Resolves #15063.

Building on top of the work @faec did in #14010, this PR allows Journalbeat inputs to take a new option, index. The value of this option determines which Elasticsearch index should be used by the Elasticsearch output for indexing Journalbeat events produced by that input. For other outputs, the index name is passed via the @metadata["raw_index"] field.

For example, with this journalbeat.yml configuration:

journalbeat.inputs:
  - paths: []
    index: foo

output.console:
  enabled: true

Running journalbeat -e will result in events like:

{"@timestamp":"2019-12-12T00:59:42.856Z","@metadata":{"beat":"journalbeat","type":"_doc","version":"8.0.0","raw_index":"foo"},...}

procs.List = append(procs.List, p)
}

func (procs *Processors) AddProcessors(p Processors) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exported method Processors.AddProcessors should have comment or be unexported

@@ -103,10 +103,25 @@ func New(config PluginConfig) (*Processors, error) {
return procs, nil
}

func (procs *Processors) add(p Processor) {
func (procs *Processors) AddProcessor(p Processor) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exported method Processors.AddProcessor should have comment or be unexported

@ycombinator ycombinator marked this pull request as ready for review December 12, 2019 19:34
Copy link
Contributor

@faec faec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@ycombinator
Copy link
Contributor Author

Travis CI is green. Jenkins CI failures are unrelated. Merging.

@ycombinator ycombinator merged commit fb4dd37 into elastic:master Dec 13, 2019
@ycombinator ycombinator deleted the jb-input-index branch December 13, 2019 22:29
ycombinator added a commit that referenced this pull request Dec 17, 2019
* Refactoring: renaming var to avoid collision

* Refactoring: extract function

* Refactoring: moving AddFormattedIndex processor into libbeat

* Add constructor for addFormattedIndex processor

* Export processor struct

* Add comment for exported constructor

* Adding comment for exported method

* Adding index option to doc

* Refactoring: extracting common code

* Adding unit tests

* Adding godoc for exported methods
@ycombinator ycombinator removed the needs_backport PR is waiting to be backported to other branches. label Dec 17, 2019
@andresrc andresrc added the Team:Integrations Label for the Integrations team label Mar 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Journalbeat] Allow inputs to specify output index
4 participants