Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flow output not paired up precisely #75

Open
codefromthecrypt opened this issue Jul 5, 2016 · 8 comments
Open

flow output not paired up precisely #75

codefromthecrypt opened this issue Jul 5, 2016 · 8 comments

Comments

@codefromthecrypt
Copy link

Looking at the flow data from cassandra, I noticed a mismatch on bookend annotations.

For example, something is logging ["cs", "sr"] on a root span. This is a bit odd, as root spans either start with a "cs" or an "sr", and end (respectively) with an "cr" or "ss". This leaves the following normal combinations, none of which being ["cs", "sr"].

["cs", "cr"]
["sr", "ss"]
["cs", "sr", "ss", "cr"]

Reason I mention this is that I'm working on cassandra tuning of zipkin, and the input data having root spans like ["cs", "sr"] leaves no means to accurately compute the span's duration (since neither the client was closed, nor the server opened). If we can change the code to generate one of the above forms, the test I'm doing will be more realistic.

{"traceId":"0000000000000546","name":"Put","id":"0000000000000546","annotations":[{"endpoint":{"serviceName":"cassandra.*.*..www00...www.denominator","ipv4":"54.198.0.13","port":8080},"timestamp":1460925455820459,"value":"cs"},{"endpoint":{"serviceName":"cassandra.us-east-1.*..www-elb00...www-elb.elb","ipv4":"54.198.0.14","port":8080},"timestamp":1460925455820515,"value":"sr"}]}
@hubayirp
Copy link

hubayirp commented Jul 5, 2016

But,

"There are four annotations the one span must have in order to construct a
full-view of a RPC call (in chronological order): cs, sr, ss, cr,
in which c stands for the client, s stands for the server and the
second s stands for send, the second r stands for receive. Please note
that these annotations does not have to be all present in one span"

On Mon, Jul 4, 2016 at 8:21 PM, Adrian Cole notifications@github.com
wrote:

Looking at the flow data from cassandra, I noticed a mismatch on bookend
annotations.

For example, something is logging ["cs", "sr"] on a root span. This is a
bit odd, as root spans either start with a "cs" or an "sr", and end
(respectively) with an "cr" or "ss". This leaves the following normal
combinations, none of which being ["cs", "sr"].

["cs", "cr"]
["sr", "ss"]
["cs", "sr", "ss", "cr"]

Reason I mention this is that I'm working on cassandra tuning of zipkin,
and the input data having root spans like ["cs", "sr"] leaves no means to
accurately compute the span's duration (since neither the client was
closed, nor the server opened). If we can change the code to generate one
of the above forms, the test I'm doing will be more realistic.

{"traceId":"0000000000000546","name":"Put","id":"0000000000000546","annotations":[{"endpoint":{"serviceName":"cassandra....www00...www.denominator","ipv4":"54.198.0.13","port":8080},"timestamp":1460925455820459,"value":"cs"},{"endpoint":{"serviceName":"cassandra.us-east-1.*..www-elb00...www-elb.elb","ipv4":"54.198.0.14","port":8080},"timestamp":1460925455820515,"value":"sr"}]}


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#75, or mute the thread
https://github.com/notifications/unsubscribe/ACWGWAs2p9iayCRZ2v_S0XATjABGydWkks5qSc3FgaJpZM4JEv82
.

@codefromthecrypt
Copy link
Author

@hubayirp I'm aware of what the annotations are used for, I'm just curious what your comment is eluding to.

Can you be more precise with your recommendation?

@adrianco
Copy link
Owner

adrianco commented Jul 5, 2016

In Spigo, there is no acknowledgement of a Put. That might be the issue.

It you look at a GetRequest/GetResponse you should see the full set of four
annotations.

Adrian

On Monday, July 4, 2016, Adrian Cole notifications@github.com wrote:

@hubayirp https://github.com/hubayirp I'm aware of what the annotations
are used for, I'm just curious what your comment is eluding to.

Can you be more precise with your recommendation?


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#75 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAG0VCzfkxf2odplG7mBFTRyvF4iapbMks5qSdnJgaJpZM4JEv82
.

@codefromthecrypt
Copy link
Author

ic, so this is about a fire-and-forget operation. duration is tricky to derive since we can't derive the time spent doing the Put from annotations. One way is we could fabricate the time spent doing the put (ex adding span.duration). Another way is for me to find another file to look at :)

@codefromthecrypt
Copy link
Author

seems to also be the case in a root span of type GetRequest in netflixoss_flow.json. I'd imagine the trip back to be in the same span.

  {
    "traceId": "0000000000000292",
    "name": "GetRequest",
    "id": "0000000000000292",
    "annotations": [
      {
        "endpoint": {
          "serviceName": "netflixoss.*.*..www00...www.denominator",
          "ipv4": "54.198.0.23",
          "port": 8080
        },
        "timestamp": 1463234805460220,
        "value": "cs"
      },
      {
        "endpoint": {
          "serviceName": "netflixoss.us-east-1.*..www-elb00...www-elb.elb",
          "ipv4": "54.198.0.24",
          "port": 8080
        },
        "timestamp": 1463234807506919,
        "value": "sr"
      }
    ]
  }

@adrianco
Copy link
Owner

adrianco commented Jul 5, 2016

The getresponse should be the same traceid to complete the flow

I could generate a response for a put, it's needed to simulate failures to
write.

Adrian

On Monday, July 4, 2016, Adrian Cole notifications@github.com wrote:

seems to also be the case in a root span of type GetRequest in
netflixoss_flow.json. I'd imagine the trip back to be in the same span.

{
"traceId": "0000000000000292",
"name": "GetRequest",
"id": "0000000000000292",
"annotations": [
{
"endpoint": {
"serviceName": "netflixoss....www00...www.denominator",
"ipv4": "54.198.0.23",
"port": 8080
},
"timestamp": 1463234805460220,
"value": "cs"
},
{
"endpoint": {
"serviceName": "netflixoss.us-east-1.*..www-elb00...www-elb.elb",
"ipv4": "54.198.0.24",
"port": 8080
},
"timestamp": 1463234807506919,
"value": "sr"
}
]
}


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#75 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAG0VCNnBIYsBo4VotVU20HzqCR7xVkfks5qSd9ZgaJpZM4JEv82
.

@codefromthecrypt
Copy link
Author

codefromthecrypt commented Jul 5, 2016 via email

@adrianco
Copy link
Owner

adrianco commented Jul 5, 2016

Ok. I'll take a look.

On Monday, July 4, 2016, Adrian Cole notifications@github.com wrote:

maybe something isn't emitting the GetResponse events to the flow json
recorder thing for the root span. I spot checked a couple, and they don't
seem to have the closing event.

https://github.com/adrianco/spigo/blob/master/json_metrics/netflix_flow.json
https://github.com/adrianco/spigo/blob/master/json_metrics/lamp_flow.json


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#75 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAG0VE06G_VTv6Z88lRGF0hbm_oGMTssks5qSfySgaJpZM4JEv82
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants