Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Langfuse: Create a simple way to add evaluation pipeline trace to trace of pipeline to get results #890

Open
mrm1001 opened this issue Jul 10, 2024 · 0 comments
Labels
feature request Ideas to improve an integration

Comments

@mrm1001
Copy link

mrm1001 commented Jul 10, 2024

Is your feature request related to a problem? Please describe.

At the moment, users need to run 2 pipelines to get evaluation metrics: the pipeline that iterates over the documents to get retrieved contexts and predicted answers, and a second pipeline to get the evaluation metrics. All the metrics in the library have individual, as well as aggregated scores. It would be great if there could be a simple way to visualise the trace for the specific RAG query alongside the evaluation pipeline trace for that query.

Describe the solution you'd like

So for example, I would like to have the trace for a specific RAG query together with the trace for the ExactAnswerMatch evaluator run on the same query/answer.

@mrm1001 mrm1001 added the feature request Ideas to improve an integration label Jul 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Ideas to improve an integration
Projects
None yet
Development

No branches or pull requests

1 participant