Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tracking] Use high resolution time in Jaeger clients #1436

Closed
1 task
yurishkuro opened this issue Mar 20, 2019 · 8 comments
Closed
1 task

[tracking] Use high resolution time in Jaeger clients #1436

yurishkuro opened this issue Mar 20, 2019 · 8 comments
Labels
help wanted Features that maintainers are willing to accept but do not have cycles to implement meta-issue An tracking issue that requires work in other repos

Comments

@yurishkuro
Copy link
Member

yurishkuro commented Mar 20, 2019

In most Jaeger clients we are using the default timer to capture timestamps, which is often only accurate to milliseconds. Most languages provide nanosecond-resolution timers, but those can only be used to measure durations, not get the wall clock timestamps. It is possible to make those work with the default timer by using the following technique

  • on the first span in process for a given trace, capture

    • the timestamp T1 using default timer. It will have millisecond precision, but that's ok since we normally expect a clock skew between processes anyway
    • the initial value t1 of the high resolution timer
  • keep the reference to these two values in all other spans started in the process for this trace

  • every time a new timestamp needs to be captured, use the current value t of the high resolution timer and calculate the new timestamp T = T1 + (t - t1)

  • Node.js client Investigate high resolution timers jaeger-client-node#31

@yurishkuro yurishkuro added meta-issue An tracking issue that requires work in other repos client-libs labels Mar 20, 2019
@m1o1
Copy link

m1o1 commented Mar 20, 2019

We were thinking about a similar calculation actually, but stumbled upon this:
https://github.com/wadey/node-microtime
which I guess uses a native node addon to provide the time.

@vprithvi
Copy link
Contributor

Could you tie this to a use case? I'm not a fan of the technique proposed as it adds precision without adding accuracy.

@m1o1
Copy link

m1o1 commented Mar 20, 2019

We use a redis cache and some spans are < 1 ms, which gets rounded to 0 and causes start times to be the same. This results in spans that should show up after this span as showing up before it in the Jaeger UI and is a difficult case to handle in the tools we're making for analyzing these traces.

@yurishkuro
Copy link
Member Author

The use case is measuring spans with sub-millisecond durations, which is quite possible on fast networks. It has been asked on a number of occasions across different repos and chat (but I am too lazy to go hunt for references).

@vprithvi
Copy link
Contributor

The use case is measuring spans with sub-millisecond durations

@yurishkuro Wouldn't durations only require high resolution timers (as opposed to high resolution wall clock timestamps)

@yurishkuro
Copy link
Member Author

That can be the first step, which is indeed much easier to implement, but it would still create misleading Gantt chart for such short traces since all span.startTime's will be clamped to 1ms.

@yurishkuro yurishkuro added the help wanted Features that maintainers are willing to accept but do not have cycles to implement label Mar 20, 2019
@Falco20019
Copy link

In C#, this highly depends on OS version and .NET runtime. Around the end of 2017, it got improved to be precise about 0.1us. Before we were at around 15ms or 0.2ms when using workarounds.

So using .NET Standard and .NET Core 2.1+ on a current platform (Unix or Windows) should already have that accuracy available in our current implementation.

@yurishkuro
Copy link
Member Author

Per #3362, we're sunsetting Jaeger clients.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Features that maintainers are willing to accept but do not have cycles to implement meta-issue An tracking issue that requires work in other repos
Projects
None yet
Development

No branches or pull requests

5 participants