time: use monotonic clock to measure elapsed time

[tsuna]

Go's standard library doesn't provide any API to access a monotonic clock source, which means one can't reliably time how long an operation takes without copy-pasting or importing some low-level platform-dependent code such as the clock package at https://github.com/davecheney/junk.

This API doesn't necessarily need to return a time.Time, just returning a number of nanoseconds as a uint64 would be enough, kinda like Java's System.nanoTime().

[adg]

Do we have a monotonic clock source we can export?

If not, why should this be in the standard library and not just an
importable package?

On 13 October 2015 at 12:42, Benoit Sigoure notifications@github.com
wrote:

Go's standard library doesn't provide any API to access a monotonic clock
source, which means one can't reliably time how long an operation takes
without copy-pasting or importing some low-level platform-dependent code
such as the clock package at https://github.com/davecheney/junk.

This API doesn't necessarily need to return a time.Time, just returning a
number of nanoseconds as a uint64 would be enough, kinda like Java's
System.nanoTime()
http://docs.oracle.com/javase/8/docs/api/java/lang/System.html#nanoTime--
.

—
Reply to this email directly or view it on GitHub
#12914.

[tsuna]

Why should this be in the standard library: because it's a pretty fundamental thing, especially in a language geared towards systems programming. I see a lot of people timing how long operations take using time.Now() and subtracting two time.Time objects, and I always end up saying "this is not a correct way of measuring how much time elapsed" but don't have a clear alternative to propose.

Come on, even Java has it.

[adg]

I see a lot of people timing how long operations take using time.Now() and subtracting two time.Time objects, and I always end up saying "this is not a correct way of measuring how much time elapsed"

Got any concrete examples?

[tsuna]

Any concrete examples of ..? People using time.Now() or it being wrong because it doesn't take into account changes made to the system time?

[adg]

I suppose what I'm getting at is this: If there's a broad need for such a package, why isn't there one that people are already using?

[extemporalgenome]

This API doesn't necessarily need to return a time.Time, just returning a number of nanoseconds as a uint64 would be enough, kinda like Java's System.nanoTime().

time.Duration happens to represent a 64-bit quantity nanoseconds, and, unless there's some large problem with it, should be favored as the return type for any monotonic duration handling functions.

[taruti]

I think there are two things:

A) A proper monotonic clock package (there are multiple 3rd party ones)

B) Measuring elapsed time on systems where system time may leap to either direction

B is simpler. Either ensure and document that time.Since works
even with clock shifts or define new API like:

type MeasureStart struct { ... }
func (*MeasureStart)Start()
func (*MeasureStart)Elapsed() time.Duration

[crawshaw]

On what platforms is time.Now() not monotonic? For linux/amd64, time.Now is implemented by runtime·nanotime, which is calling clock_gettime (via VDSO) with CLOCK_MONOTONIC. I see similar calls being used on linux/arm, openbsd, freebsd, etc.

Instead of a new interface, can we document time.Now as monotonic?

[ianlancetaylor]

@crawshaw time.Now is not monotonic on any platform that supports settimeofday. On amd64 GNU/Linux time.Now does not actually use CLOCK_MONOTONIC, it uses CLOCK_REALTIME. It's runtime.nanotime that uses CLOCK_MONOTONIC.

[ianlancetaylor]

Sorry, I somehow missed that you mentioned nanotime yourself. That's not what implements time.Now, though, that is implemented by time·now.

[crawshaw]

Ah right, thanks. I got confused by //go:linkname time_runtimeNano time.runtimeNano, which I misread as implementing time.Now

[alexbrainman]

time.Now() is monotonic on windows. We use undocumented way to fetch time, but it is monotonic as far as I know. @dvyukov to confirm.

Alex

[tsuna]

On Mon, Oct 12, 2015 at 9:07 PM, Andrew Gerrand wrote:

I suppose what I'm getting at is this: If there's a broad need for such a package, why isn't there one that people are already using?

Because people incorrectly assume that time only moves forward? It's an extremely common mistake that even at Google we were trying to hammer in people's head. When you see the large impact that things such as leap seconds have (including at Google), it's easy to see this is a well known problem but also a recurring problem.

How is it that a language geared towards systems programming that comes with such a rich standard library offers no standard way of measuring time correctly?

Look at gRPC, every single use of time.Now() there is buggy because it implicitly assumes that time only moves forward and at a constant pace, neither of which is true for CLOCK_REALTIME. Pretty much any distributed systems code that uses time.Now() other than for printing the current time is subtly buggy. And there is a lot of it. And it's there chiefly because there is no alternative in the standard library to get monotonic time so people don't even think about it.

[rsc]

How is it that a language geared towards systems programming that comes with such a rich standard library offers no standard way of measuring time correctly?

I expect that if you care that much about time you will run your system clocks correctly. On a well-run system, time does only move forward and at a constant (enough) pace.

When you see the large impact that things such as leap seconds have (including at Google), it's easy to see this is a well known problem but also a recurring problem.

I don't understand this comment. Leap seconds don't exist at Google.

It is true that if you use time.Now to measure time during a leap smear then your measurements may be off by approximately 0.001%. But time will still be monotonic.

[dvyukov]

I expect that if you care that much about time you will run your system clocks correctly. On a well-run system, time does only move forward and at a constant (enough) pace.

How about programs that run on client machines? Or server software that you sells to clients?

[dvyukov]

time.Now() is monotonic on windows. We use undocumented way to fetch time, but it is monotonic as far as I know. @dvyukov to confirm.

I don't think we return monotonic time from time.Now on windows. This would be a bug, because time.Now should return system time which can be changed by user.
The shared page contains InterruptTime, this time is monotonic and we use it for runtime.nanotime; and SystemTime, this time is not monotonic and we use for time.Now.

[alexbrainman]

What Dmitry said.

Alex

[tsuna]

On Fri, Oct 23, 2015 at 9:50 PM, Russ Cox wrote:

I don't understand this comment. Leap seconds don't exist at Google.

I was working just down the hall from where you were in B43 in 2008, when the leap second smearing scheme was devised, because everybody remembered all too well how all hell broke loose for the previous leap second.

[driskell]

I'm not quite understanding, why the time and net package can get access to a monotonic clock, through runtime package, but nothing else can. Surely it would be beneficial to allow others to write packages that have similar capabilities to the time and net standard libraries? It seems the protections of the monotonic clock are currently limited to the standard library and inaccessible to all other libraries.

I can understand the arguments about clocks should never go backwards but the fact that time and net is already using monotonic time, in my opinion, means that discussion was had a long time ago and the argument for monotonic time won out. It's unfortunate it's been kept inaccessible to everyone else.

Is it feasible and would it be acceptable to expose a runtime.NanoTime? That name is likely misleading though but it seems reasonable to me. (https://github.com/golang/go/blob/master/src/runtime/time.go#L296)

[tsuna]

For those who need a solution to this problem, here's a way to expose Go's own runtime.nanotime(), which avoids the hassle of reimplementing an Nth solution that is fast & cross-platform: aristanetworks/goarista@46272bf

[cespare]

Looks like the current code on master uses 1885-2157 as the range of wall times when monotonic time is present.

[rsc]

OK, this is checked in for Go 1.9. I moved the internal epoch to 1885 (max year 2157) to avoid any possible problem with NTP-epoch-derived times. I also adjusted the design doc to reflect this change and some minor encoding changes.

Thanks for the constructive conversation everyone.

[orivej]

1. The merged implementation changes the meaning of time.Now().Sub(old) when now and old are separated by system suspend and resume: it used to calculate real time difference and now that difference is reduced by the duration the system was suspended. Shouldn't time use monotonic real time source when available, and if so, can runtime.nanotime be redefined to read from that source, or should there be a new clock specifically for time.Now() to read from?

2. Is there an explanation for + 1 in nanotime() - startNano + 1 in the design doc? If it is needed, why startNano is not reduced by 1 instead?

[rsc]

@orivej,

1. Whether time.Since(start) includes "system suspended" time depends on the underlying OS's monotonic clock, but in general it's tricky to add that time into the clock since you can't take it back if you later find out you are wrong. My preference here would be to do the same thing on all operating systems, whatever that is. My guess is that all operating systems can provide the current semantics but that not all can provide the "include suspended time" semantics.

   It seems to me that if you want to include time while the computer was off, you're really talking about wall times, so the fix would be to flip into wall mode by recording 'start := time.Now().AddDate(0, 0, 0)', putting up with the fact that time.Since(start) might then return negative elapsed times.

2. The +1 is just to make sure that tests never see a 0 monotonic time, since I used GetMono(&t) == 0 to say that there's no monotonic time. It's only for testing, and even in that context it's probably unnecessary, but also harmless.

[gopherbot]

CL https://golang.org/cl/36334 mentions this issue.

[neild]

I'm looking at some code which assumes that times truncated to the same time are equal. Simplified:

func main() {
        t1 := time.Now().Truncate(1 * time.Second)
        t2 := time.Now().Truncate(1 * time.Second)
        h1, m1, s1 := t1.Clock()
        h2, m2, s2 := t2.Clock()
        if h1 == h2 && m1 == m2 && s1 == s2 {
                if !t1.Equal(t2) {
                        fmt.Printf("BUG: times truncated to the same second, but are not equal:\n%v\n%v\n", t1, t2)
                }
        }
}

The actual code is mapping times to windows; it assumes that all times within a given window will truncate to the same time.

This doesn't seem unreasonable. Should Truncate discard the monotonic time component?

[dmitshur]

@neild A relevant open issue is #18991.