Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use RFC3339 in UTC as the default time format for log messages #84

Merged
merged 3 commits into from
Dec 2, 2024

Conversation

efd6
Copy link
Contributor

@efd6 efd6 commented Nov 17, 2024

This ensures that times that are logged in messages can be deserialised by the encoding/json package irrespective of the timezone that the time is in when logged; currently if the time is not in UTC, the ISO8601 format that is used is not accepted. Also provide a non-UTC formatter that can be used where the timezone is important.

@efd6 efd6 force-pushed the timestamp_rfc3339 branch from 3a2c871 to f120a05 Compare November 17, 2024 21:33
@efd6 efd6 marked this pull request as ready for review November 17, 2024 21:37
@efd6 efd6 force-pushed the timestamp_rfc3339 branch from f120a05 to 084b072 Compare November 17, 2024 21:43
@belimawr belimawr added the Team:Elastic-Agent Label for the Agent team label Nov 18, 2024
Copy link

@belimawr belimawr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just a bit confused why we need RFC3339UTCTimeEncoder and RFC3339TimeEncoder if only RFC3339UTCTimeEncoder is used directly and the only thing it does is to call RFC3339TimeEncoder with the time in UTC, which is the first thing RFC3339TimeEncoder does.

@efd6 could you clarify what is the effective difference between those two functions?

@efd6
Copy link
Contributor Author

efd6 commented Nov 18, 2024

That is a bug.

This ensures that times that are logged in messages can be deserialised
by the encoding/json package irrespective of the timezone that the time
is in when logged; currently if the time is not in UTC, the ISO8601
format that is used is not accepted. Also provide a non-UTC formatter
that can be used where the timezone is important.
@efd6 efd6 force-pushed the timestamp_rfc3339 branch from 084b072 to 93faa27 Compare November 18, 2024 20:48
@efd6
Copy link
Contributor Author

efd6 commented Nov 18, 2024

Fixed. Thanks.

Copy link
Member

@kruskall kruskall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Can you add a test ?

@efd6
Copy link
Contributor Author

efd6 commented Nov 21, 2024

There are already tests for this.

@kruskall
Copy link
Member

Sorry, I might be missing something

currently if the time is not in UTC, the ISO8601 format that is used is not accepted.

where is this tested ?

@efd6
Copy link
Contributor Author

efd6 commented Nov 21, 2024

That's specified here.

@efd6 efd6 requested a review from kruskall November 26, 2024 02:11
@axw
Copy link
Member

axw commented Dec 2, 2024

I think @kruskall is thinking of a test that the default timestamp encoder is able to accept non-UTC timestamps, for this library.

Put another way: can we add a test that would fail without the change in this PR?

@axw
Copy link
Member

axw commented Dec 2, 2024

@efd6 how about this?

diff --git a/logger_test.go b/logger_test.go
index 5f08f74..c72335c 100644
--- a/logger_test.go
+++ b/logger_test.go
@@ -77,10 +77,13 @@ func (tw *testOutput) validate(t *testing.T, keys ...string) {
 			case "string":
 				_, ok = val.(string)
 			case "datetime":
-				var s string
-				s, ok = val.(string)
-				if _, err := time.Parse("2006-01-02T15:04:05.000Z0700", s); err == nil {
-					ok = true
+				if s, strOK := val.(string); strOK {
+					if _, err := time.Parse("2006-01-02T15:04:05.000Z0700", s); err == nil {
+						var t time.Time
+						if err := t.UnmarshalText([]byte(s)); err == nil {
+							ok = true
+						}
+					}
 				}
 			case "integer":
 				// json.Unmarshal unmarshals into float64 instead of int
@@ -160,3 +163,29 @@ func TestECSZapLogger(t *testing.T) {
 		})
 	}
 }
+
+func TestECSZapLoggerTimezone(t *testing.T) {
+	out := testOutput{}
+
+	eucla, err := time.LoadLocation("Australia/Eucla")
+	require.NoError(t, err)
+
+	core := NewCore(NewDefaultEncoderConfig(), &out, zap.DebugLevel)
+	logger := zap.New(core, zap.WithClock(&locationClock{
+		Clock:    zapcore.DefaultClock,
+		location: eucla,
+	}))
+	defer logger.Sync()
+
+	logger.Info("hello from UTC+8:45")
+	out.validate(t, "@timestamp")
+}
+
+type locationClock struct {
+	zapcore.Clock
+	location *time.Location
+}
+
+func (c *locationClock) Now() time.Time {
+	return c.Clock.Now().In(c.location)
+}

Also fix time validation in logger tests.
@efd6
Copy link
Contributor Author

efd6 commented Dec 2, 2024

The time validation in validate is not right; it assumes the incorrect ISO8601 format. I've gone with a behaviour-based test that looks to whether json.Unmarshal cope with it only.

The config test is enough to show that the change is required.

Copy link
Member

@axw axw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for adding the test, looks good

logger_test.go Show resolved Hide resolved
@axw axw enabled auto-merge (squash) December 2, 2024 05:56
@axw axw merged commit 43fc05e into elastic:main Dec 2, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Elastic-Agent Label for the Agent team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants