-
Notifications
You must be signed in to change notification settings - Fork 17.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
runtime: nanosecond precision lost since mac os x high sierra update #22037
Comments
This implies that High Sierra has changed the implementation of the commpage described in http://www.opensource.apple.com/source/xnu/xnu-1699.26.8/osfmk/i386/cpu_capabilities.h in such a way that the Go code (line 109 of https://tip.golang.org/src/runtime/sys_darwin_amd64.s) falls back to calling Can you find the cpu_capabilities.h file that applies to High Sierra? Or post a disassembly of the It would also be useful to know whether the runtime is fetching useful monotonic time readings. Can you I very much hope this is not another #16570. |
@ianlancetaylor Here is the ouptut for 2017-09-26 20:48:07.875345 +0530 IST m=+0.000358664 |
Thanks. My vague guess is that the time information on the commpage is present but the generation field is always zero. Or something. The other info I requested might tell us. |
@ianlancetaylor Here is the dump of the clock_gettime function from the lib. I also saw some clock_gettime_nsec_np and included that too. |
Thanks! Unfortunately the key steps seem to be in a function called via |
@ianlancetaylor Sure.. here you go. It is btw the |
Thanks. Now it looks like I need something that appears in a different library. Looking at the older xnu sources (I can't find the 10.13 sources online anywhere) it looks like I might need libsyscall. Do you have that library? |
@ianlancetaylor sorry I dont seem to find that library.. Can you give me some pointers on finding it. |
Sorry, based on Mac OS 10.12 the library I'm looking for is called libsys_kernel.dylib. In particular I'm looking for the functions |
Attached |
Thanks for all the info. The Here is an annotated disassembly of
|
I dumped values from my Mac right now (~01:36 CEST): 0xD0: d1cc511a4d35 (gtod_ns_base) |
Updated with more values (~01:42 CEST): 0xD0: d21b247d1256 (gtod_ns_base) |
doesn't i'm a bit rusty with my assembly. but, after multiplying by ns_scale (at 130ea/130f0), the upper 64bits (rdx register) is used as the "seconds" part of the time. this indicates to me that the units after the multiplication is not sure if i missed anything from the assembly but here's the c code i used to try and get a better grasp of it.
|
so python is able to do this just fine with time.time() by making it into a float
but I don't think it's a fix
there's also the clock_gettime_nsec_np function but it has the same problem as time.Now().UnixNano() |
Relevant snippet from XNU sources: https://opensource.apple.com/source/xnu/xnu-4570.1.46/osfmk/i386/cpu_capabilities.h
https://opensource.apple.com/source/xnu/xnu-4570.1.46/bsd/sys/commpage.h
https://opensource.apple.com/source/xnu/xnu-4570.1.46/libsyscall/wrappers/__commpage_gettimeofday.c
|
I'm working on a fix for this |
Change https://golang.org/cl/67331 mentions this issue: |
Change https://golang.org/cl/67332 mentions this issue: |
Python is invoking syscalls that return values with microsecond accuracy. The fact that they get then changed into float is immaterial, there is still no nanosecond accuracy. Basically Python does what Go happens to do now on High Sierra: fallback to the slower path (syscalls) with lower accuracy. Go accesses the so-called commpage, which is a special area that the kernel maps into every process at the same virtual address, and contains useful information that can be accessed directly without the overhead of a syscall. In High Sierra, the commpage format has been bumped (from 12 to 13) changing the way the wall/real time is returned; the monotonic time is not changed. So what happens right now is that time.Now() on current Go versions is able to read the monotonic timer with nanosecond precision, but then fails to convert it to real time; it does realize there is something wrong, and falls back to calling a syscall to get the real time (but with microsecond precision). My CL adds support for detecting commpage v13, and implements the new required formulas to convert the monotonic time read into real time, with full nanosecond accuracy. The assembly code is basically the equivalent of the above snippet of code coming from XNU sources, but it's made easier by the fact that we can do some 128-bit calculations in Assembly, while it would not possible in Go, nor it is possible in the dialect of C used for XNU (some GCC versions do support 128-bit calculations on some platforms). |
High Sierra has a new commpage layout (this is issue #3188), so we need to adjust the code to handle multiple versions of the layout. In preparation for this change, we rename the existing offset macros with a prefix that identifies the commpage version they refer to. Updates #22037 Change-Id: Idca4b7a855a2ff6dbc434cd12453fc3194707aa8 Reviewed-on: https://go-review.googlesource.com/67331 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reopen for 1.9 backport. |
This change is a lot of new assembly and scares me. The old code was correctly falling back to calling into the kernel to get the time, which seems OK enough. I'm not even convinced the nanoseconds are accurate on macOS anyway - last time I looked, the kernel was setting the "call into the kernel anyway" bit in the commpage about once a second to slow down time because the commpage version of time was running too fast, consistently. So the nanoseconds were probably racing ahead quite a bit. Can we leave this out of Go 1.9.2? |
I think it's ok not to backport it, and reevaluate the decision if somebody comes screaming with real code being broken. Given that time measures use monotonic support, I don't think that real-time nanoseconds are that useful.
I don't see that happening now but I'm not sure I'm testing it correctly. |
@rasky, thanks for pointing out that monotonic time is unaffected. Precise timing of small intervals was the only thing I was worried about, so with that unaffected I agree this can wait. |
Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (
go version
)?go1.9 darwin/amd64
Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (
go env
)?Mac OS X High Sierra
What did you do?
fmt.Println(time.Now().UnixNano())
What did you expect to see?
i would expect to display amount of nanoseconds
What did you see instead?
nanosecond part is not filled
some results
1506416154931120000
1506416154931136000
1506416154931138000
1506416154931139000
1506416154931140000
The text was updated successfully, but these errors were encountered: