You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently Opik allows only 2 types of annotations: categorical and numerical. I propose to add a third type: free text.
Motivation
This would allow non-technical domain experts to evaluate the output of each step in the prompting process for a dataset sample, and highlight where the model made a wrong decision. Currently this is not possible, and this type of evaluation has to be made in other tools (e.g. spreadsheets).
The text was updated successfully, but these errors were encountered:
Proposal summary
Currently Opik allows only 2 types of annotations: categorical and numerical. I propose to add a third type: free text.
Motivation
This would allow non-technical domain experts to evaluate the output of each step in the prompting process for a dataset sample, and highlight where the model made a wrong decision. Currently this is not possible, and this type of evaluation has to be made in other tools (e.g. spreadsheets).
The text was updated successfully, but these errors were encountered: