feat(traces): server-side sort of spans by evaluation result (score or label) #1812
Conversation
| col: SpanColumn = null | ||
| evalResultKey: EvalResultKey = null |
There was a problem hiding this comment.
oneOf for input types is not available in strawberry
| def __call__( | ||
| self, | ||
| span: Span, | ||
| evals: Optional[SupportsGetSpanEvaluation] = None, | ||
| ) -> Any: | ||
| """ | ||
| Returns the evaluation result for the given span | ||
| """ | ||
| if evals is None: | ||
| return None | ||
| span_id = span.context.span_id | ||
| evaluation = evals.get_span_evaluation(span_id, self.name) | ||
| if evaluation is None: | ||
| return None | ||
| result = evaluation.result | ||
| if self.attr is EvalAttr.score: | ||
| return result.score.value if result.HasField("score") else None | ||
| if self.attr is EvalAttr.label: | ||
| return result.label.value if result.HasField("label") else None | ||
| assert_never(self.attr) |
There was a problem hiding this comment.
minor nit on semantics: making the key a callable rather than having an explicit method that has a well-formed name is confusing for me. For example the code below doesn't reed well to me - the key appears to be a simple param but then it's getting partially called with evals. It forces me to have to go to definition to understand what calling the key does, which is a level of indirection that I think you can avoid by just having a good name for a method or function.
_key = partial(self.eval_result_key, evals=evals)There was a problem hiding this comment.
I can change it. Not a problem. But will just note the reason why it was done.
keyis a callable in both python and pandas, so it's part of the python tradition.- python functions are objects, so initializing it with parameters alludes to python's functional paradigm.
- it's fun since it's a programmer's version of a double entendre (given 1 and 2 above)
There was a problem hiding this comment.
I see, yeah I think Key being a callable makes sense but in this case it requires a partial application to work. But this makes sense. Your call. Thanks for the explanation.
There was a problem hiding this comment.
No problem. I already changed it.
…r label) (#1812) * feat: server-side sort by eval
resolves #1776
Screen.Recording.2023-11-27.at.4.09.17.PM.mov