Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add processor to set timezone for an event #3902

Merged
merged 13 commits into from
Apr 7, 2017
1 change: 1 addition & 0 deletions CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ https://github.com/elastic/beats/compare/v5.1.1...master[Check the HEAD diff]
- Disabled date detection in Elasticsearch index templates. Date fields must be explicitly defined in index templates. {pull}3528[3528]
- Using environment variables in the configuration file is now GA, instead of experimental. {pull}3525[3525]
- Initialize a beats UUID from file on startup. {pull}3615[3615]
- Add new `add_locale` processor to export the local timezone with an event. {pull}3902[3902]

*Filebeat*

Expand Down
5 changes: 5 additions & 0 deletions filebeat/filebeat.full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -419,6 +419,11 @@ filebeat.prospectors:
#processors:
#- add_cloud_metadata:
#
# The following example enriches each event with the local timezone.
#
#processors:
#- add_locale:
#

#================================ Outputs ======================================

Expand Down
5 changes: 5 additions & 0 deletions heartbeat/heartbeat.full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,11 @@ heartbeat.scheduler:
#processors:
#- add_cloud_metadata:
#
# The following example enriches each event with the local timezone.
#
#processors:
#- add_locale:
#

#================================ Outputs ======================================

Expand Down
5 changes: 5 additions & 0 deletions libbeat/_meta/config.full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,11 @@
#processors:
#- add_cloud_metadata:
#
# The following example enriches each event with the local timezone.
#
#processors:
#- add_locale:
#

#================================ Outputs ======================================

Expand Down
4 changes: 4 additions & 0 deletions libbeat/_meta/fields.common.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@
description: >
The hostname as returned by the operating system on which the Beat is
running.
- name: beat.timezone
description: >
The timezone as returned by the operating system on which the Beat is
running.
- name: beat.version
description: >
The version of the beat that generated this event.
Expand Down
1 change: 1 addition & 0 deletions libbeat/beat/beat.go
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ import (
// Register default processors.
_ "github.com/elastic/beats/libbeat/processors/actions"
_ "github.com/elastic/beats/libbeat/processors/add_cloud_metadata"
_ "github.com/elastic/beats/libbeat/processors/add_locale"

// Register default monitoring reporting
_ "github.com/elastic/beats/libbeat/monitoring/report/elasticsearch"
Expand Down
18 changes: 18 additions & 0 deletions libbeat/docs/processors-config.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,7 @@ not:
The supported processors are:

* <<add-cloud-metadata,`add_cloud_metadata`>>
* <<add-locale,`add_locale`>>
* <<decode-json-fields,`decode_json_fields`>>
* <<drop-event,`drop_event`>>
* <<drop-fields,`drop_fields`>>
Expand Down Expand Up @@ -340,6 +341,23 @@ _GCE_
}
-------------------------------------------------------------------------------

[[add-locale]]
=== add_locale

The `add_locale` processor enriches each event with the machine's time zone.

The simple configuration below enables the processor.

[source,yaml]
-------------------------------------------------------------------------------
processors:
- add_locale:
-------------------------------------------------------------------------------

NOTE: Please consider that `add_locale` differentiates between DST and regular time.
For example `CET` and `CEST`.


[[decode-json-fields]]
=== decode_json_fields

Expand Down
34 changes: 34 additions & 0 deletions libbeat/processors/add_locale/add_locale.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
package actions

import (
"time"

"github.com/elastic/beats/libbeat/common"
"github.com/elastic/beats/libbeat/processors"
)

type addLocale struct {
timezone string
}

func init() {
processors.RegisterPlugin("add_locale", newAddLocale)
}

func newAddLocale(c common.Config) (processors.Processor, error) {
zone, _ := time.Now().Zone()

l := addLocale{timezone: zone}

return l, nil
}

func (l addLocale) Run(event common.MapStr) (common.MapStr, error) {
event.Put("beat.timezone", l.timezone)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the requirement to have this field automatically reflect the latest timezone, let's not cache the timezone like I requested previous (sorry for the change) and make Run always call time.Now().Zone(). Based on the benchmark results there wasn't a huge cost to doing this.


return event, nil
}

func (l addLocale) String() string {
return "add_locale=" + l.timezone
}
44 changes: 44 additions & 0 deletions libbeat/processors/add_locale/add_locale_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
package actions

import (
"testing"
"time"

"github.com/elastic/beats/libbeat/common"
"github.com/elastic/beats/libbeat/logp"
"github.com/stretchr/testify/assert"
)

func TestExportTimeZone(t *testing.T) {
var testConfig = common.NewConfig()

input := common.MapStr{}

zone, _ := time.Now().In(time.Local).Zone()

actual := getActualValue(t, testConfig, input)

expected := common.MapStr{
"beat": map[string]string{
"timezone": zone,
},
}

assert.Equal(t, expected.String(), actual.String())
}

func getActualValue(t *testing.T, config *common.Config, input common.MapStr) common.MapStr {
if testing.Verbose() {
logp.LogInit(logp.LOG_DEBUG, "", false, true, []string{"*"})
}

p, err := newAddLocale(*config)
if err != nil {
logp.Err("Error initializing add_locale")
t.Fatal(err)
}

actual, err := p.Run(input)

return actual
}
5 changes: 5 additions & 0 deletions metricbeat/metricbeat.full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -401,6 +401,11 @@ metricbeat.modules:
#processors:
#- add_cloud_metadata:
#
# The following example enriches each event with the local timezone.
#
#processors:
#- add_locale:
#

#================================ Outputs ======================================

Expand Down
5 changes: 5 additions & 0 deletions packetbeat/packetbeat.full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -524,6 +524,11 @@ packetbeat.protocols:
#processors:
#- add_cloud_metadata:
#
# The following example enriches each event with the local timezone.
#
#processors:
#- add_locale:
#

#================================ Outputs ======================================

Expand Down
5 changes: 5 additions & 0 deletions winlogbeat/winlogbeat.full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,11 @@ winlogbeat.event_logs:
#processors:
#- add_cloud_metadata:
#
# The following example enriches each event with the local timezone.
#
#processors:
#- add_locale:
#

#================================ Outputs ======================================

Expand Down