Add strong response types for vertex evaluators#2

Merged

tagboola merged 238 commits intomainfrom ta/vertex-evals-strong-types

May 2, 2024

+60-12

Commits on May 2, 2024

Fix model routing (#161 )

MichaelDoyle
authored and
tagboola
committed
[UI] Add new span tree + viewer to Flow details page (#164 )

tonybaroneee
authored and
tagboola
committed
Fetch models from API (#174 )

MichaelDoyle
authored and
tagboola
committed
Backend errors (#163 )

MaesterChestnut
authored and
tagboola
committed
[UI] Cleanup unimplemented pages from navbar (#180 )

tonybaroneee
authored and
tagboola
committed
[UI] Increase max-height of flow input/output (#179 )

tonybaroneee
authored and
tagboola
committed
Move flow runner to Actions page (#176 )

shrutip90
authored and
tagboola
committed
[UI] Fix overflow of execution span tree (#183 )

tonybaroneee
authored and
tagboola
committed
Input validation disables prompt run button (#182 )

MaesterChestnut
authored and
tagboola
committed
Route playground from flows to action runners page (#191 )

MichaelDoyle
authored and
tagboola
committed
Switch temperature to the slider (#195 )

MichaelDoyle
authored and
tagboola
committed
Show validation errors on the playground (#196 )

MichaelDoyle
authored and
tagboola
committed
[UI] Revamp flow details page layout (#197 )

tonybaroneee
authored and
tagboola
committed
Fix validator issues (#194 )

shrutip90
authored and
tagboola
committed
[UI] Initial design of span details view (#199 )

tonybaroneee
authored and
tagboola
committed
Move flow runner to start from action-list instead of action-runner (#200 )

shrutip90
authored and
tagboola
committed
Add vertex-ai to the model playground (#201)

MichaelDoyle
authored and
tagboola
committed
[UI] Hide input/output pre if none available (#204 )

tonybaroneee
authored and
tagboola
committed
[UI] Add "muted" helper class for secondary text (#206 )

tonybaroneee
authored and
tagboola
committed
Don't send blank stop sequences to the model, vertex gemini model doesn't like it (#217 )

pavelgj
authored and
tagboola
committed
Provider specific model param restrictions on input (#224 )

MaesterChestnut
authored and
tagboola
committed
Use the minfied version of Monaco Editor in the angular app (#242 )

MichaelDoyle
authored and
tagboola
committed
[UI] Update app name

tonybaroneee
authored and
tagboola
committed
[UI] Update flow details layout (#246 )

tonybaroneee
authored and
tagboola
committed
[UI] Add callout component (#244 )

tonybaroneee
authored and
tagboola
committed
[UI] Hide wrapper spans on details page (#254 )

tonybaroneee
authored and
tagboola
committed
[UI] Update flow durations on details page (#256 )

tonybaroneee
authored and
tagboola
committed
[UI] Show error on flow details page (#258 )

tonybaroneee
authored and
tagboola
committed
Playground load trace (#262 )

MichaelDoyle
authored and
tagboola
committed
Add theme toggling for JSON editor and move schema to a tab next to the editor (#245 )

shrutip90
authored and
tagboola
committed
Give topP the slider treatment (#264 )

MichaelDoyle
authored and
tagboola
committed
[UI] Show flow name in tree (#266 )

tonybaroneee
authored and
tagboola
committed
[UI] Show span state in details pane (#268 )

tonybaroneee
authored and
tagboola
committed
[UI] Flows table style improvements (#269 )

tonybaroneee
authored and
tagboola
committed
[UI] Small flow details page improvements for narrow screens (#273 )

tonybaroneee
authored and
tagboola
committed
Add CustomOptions (#276 )

MichaelDoyle
authored and
tagboola
committed
[UI]Remove sample calls for unsupported actions. Small fixes in flow runner. (#275 )

shrutip90
authored and
tagboola
committed
Create Message sub component for ModelPlayground (#271 )

MaesterChestnut
authored and
tagboola
committed
Fix error with model not accepting request_format (#279 )

huangjeff5
authored and
tagboola
committed
Disable the minimap on the monaco editor (#286 )

MichaelDoyle
authored and
tagboola
committed
[UI] Add zero state for flows list page (#291 )

tonybaroneee
authored and
tagboola
committed
[UI] Fix ng error in flow runner (#297 )

tonybaroneee
authored and
tagboola
committed
[UI] Hide stream response checkbox for durable flows (#299 )

tonybaroneee
authored and
tagboola
committed
Integrating the Message component into the Prompt Playground

MaesterChestnut
authored and
tagboola
committed
Switch model select from native to mat-select (#306 )

MichaelDoyle
authored and
tagboola
committed
Ability to show errors on actions page (#307 )

MichaelDoyle
authored and
tagboola
committed
[UI] Revamp Actions list UI (#308 )

tonybaroneee
authored and
tagboola
committed
[UI] Remove unnecessary return (#309 )

tonybaroneee
authored and
tagboola
committed
[UI] Prevent selecting action if no param is set (#310 )

tonybaroneee
authored and
tagboola
committed
Enable support for multiple messages coming from traceId (#314 )

MaesterChestnut
authored and
tagboola
committed
Avoid making flow runner editors read only (#321 )

shrutip90
authored and
tagboola
committed
[UI] Add filtering and expand/collapse all to actions list (#319 )

tonybaroneee
authored and
tagboola
committed
Fix error where model selection does not update (#323 )

MichaelDoyle
authored and
tagboola
committed
[UI] Fix action search input style (#325 )

tonybaroneee
authored and
tagboola
committed
[UI] Update action list name and key display (#328 )

tonybaroneee
authored and
tagboola
committed
User error callout component on model playground (#330 )

MichaelDoyle
authored and
tagboola
committed
refactor the code around checking for json output support (#304 )

huangjeff5
authored and
tagboola
committed
Render images in chat (#340 )

MichaelDoyle
authored and
tagboola
committed
Functioning add and remove button (#335 )

MaesterChestnut
authored and
tagboola
committed
Refactor criteria/validation logic out of playground component (#339 )

MaesterChestnut
authored and
tagboola
committed
[UI] Flow runner UI polish + improvements (#343 )

tonybaroneee
authored and
tagboola
committed
Move JSON editor to shared components since retriever playground also needs it (#344 )

shrutip90
authored and
tagboola
committed
[UI] Small handful of UI nit fixes (#345 )

tonybaroneee
authored and
tagboola
committed
[UI] Add loading state to flows table (#349 )

tonybaroneee
authored and
tagboola
committed
Do not load output from trace; typically we're interested in loading up the inputs, and re-running to get the output (#347 )

MichaelDoyle
authored and
tagboola
committed
Make response_format optional (#350 )

MichaelDoyle
authored and
tagboola
committed
[UI] Add Genkit icon (#371 )

tonybaroneee
authored and
tagboola
committed
Reset streamed chunks when rerunning the streamed flow (#379 )

pavelgj
authored and
tagboola
committed
[UI] Add tooltips to span state icons (#351 )

tonybaroneee
authored and
tagboola
committed
Prefer includes over contains (#376 )
tagboola
committed
[UI] Add inspect flow state button if flow errors (#382 )

tonybaroneee
authored and
tagboola
committed
Chat mode (#391 )

MaesterChestnut
authored and
tagboola
committed
Ability to open Flow runner from the trace view (#394)

MichaelDoyle
authored and
tagboola
committed
Add basics of the eval runner page (#367 )

ssbushi
authored and
tagboola
committed
[UI] Fix flow details sidebar colors in dark mode (#399 )

tonybaroneee
authored and
tagboola
committed
[UI] Revamp model playground to chat-based layout (#397 )

tonybaroneee
authored and
tagboola
committed
[UI] Flow runner: Add a callout for no output so we dont show empty response boxes (#403 )

shrutip90
authored and
tagboola
committed
[UI] Add trace details view (#405)

tonybaroneee
authored and
tagboola
committed
role:system message allowed for models (#402 )

MaesterChestnut
authored and
tagboola
committed
Adds support for image models. (#426 )

mbleigh
authored and
tagboola
committed
fix playground runner after runAction change (#429 )

MaesterChestnut
authored and
tagboola
committed
Revert "fix playground runner after runAction change (#429 )" (#431 )

MaesterChestnut
authored and
tagboola
committed
Small tweaks to model playground to reduce chat (#438 )

MichaelDoyle
authored and
tagboola
committed
[UI] Update stackTraceSpans to filter out internal spans (#439 )

tonybaroneee
authored and
tagboola
committed
[UI] Add traces table to inspect index page (#448 )

tonybaroneee
authored and
tagboola
committed
Adding traces to Messages (#432 )

MaesterChestnut
authored and
tagboola
committed
[UI] Update routing for inspect pages (#449 )

tonybaroneee
authored and
tagboola
committed
[UI] Update routing for run pages (#450 )

tonybaroneee
authored and
tagboola
committed
[UI] Fix trace display name in table (#451 )

tonybaroneee
authored and
tagboola
committed
Allow size to be optional (#452 )

MichaelDoyle
authored and
tagboola
committed
[UI] Fix trace deep links in model playground (#453 )

tonybaroneee
authored and
tagboola
committed
[UI] Add raw mat-table for evals view (#430 )

ssbushi
authored and
tagboola
committed
Add embeddings models (#303 )

huangjeff5
authored and
tagboola
committed
[UI] Update /evaluations route to /evaluate (#454 )

tonybaroneee
authored and
tagboola
committed
[UI] Make all run buttons consistent in playgrounds (#455 )

tonybaroneee
authored and
tagboola
committed
[UI] Add cmd/ctrl + enter shortcut to playground editors (#456 )

tonybaroneee
authored and
tagboola
committed
[UI] Add landing state for Run page (#465 )

tonybaroneee
authored and
tagboola
committed
[UI] Prevent mat-slider from shrinking (#473 )

tonybaroneee
authored and
tagboola
committed
[UI] Adjust element widths for narrow browsers (#474 )

tonybaroneee
authored and
tagboola
committed
[UI] Prevent welcome page flicker on action refresh (#475 )

tonybaroneee
authored and
tagboola
committed
Add tab for Auth input to Flow Runner action (#467 )

MaesterChestnut
authored and
tagboola
committed
[UI] Add JSON sample to flow runner (#479 )

tonybaroneee
authored and
tagboola
committed
Generic action runner (#484 )

MichaelDoyle
authored and
tagboola
committed
[UI] Add support for tool primitive on dev UI run page (#488 )

tonybaroneee
authored and
tagboola
committed
[UI] Tighten up spacing of actions list items (#489)

tonybaroneee
authored and
tagboola
committed
[UI] Trigger change detection on flow runner response (#486)

tonybaroneee
authored and
tagboola
committed
[UI] Add cmd/ctrl + enter shortcut to model playground (#485 )

tonybaroneee
authored and
tagboola
committed
[UI] Update eval results UI to use expandable cards for results (#491 )

shrutip90
authored and
tagboola
committed
[UI] Prevent scrolling past last line in monaco editor (#495 )

tonybaroneee
authored and
tagboola
committed
[UI] Use helper class to style pre stacktrace in callout (#502 )

tonybaroneee
authored and
tagboola
committed
[UI]Evals UI: Update inputs to use a table format (#496 )

shrutip90
authored and
tagboola
committed
[UI] Model playground message styling polish (#515 )

tonybaroneee
authored and
tagboola
committed
[UI] Fix json editor to ignore initial value if no schema (#517 )

tonybaroneee
authored and
tagboola
committed
[UI] Set retriever name in playground header (#518 )

tonybaroneee
authored and
tagboola
committed
[UI] Prevent JSON sample pre-fill if unnecessary (#520 )

tonybaroneee
authored and
tagboola
committed
Remove fdescribe in tests (#532 )

ssbushi
authored and
tagboola
committed
Fix minor UI elements in eval page (#533 )

ssbushi
authored and
tagboola
committed
Add inspect trace option (#540 )

ssbushi
authored and
tagboola
committed
Add inspect tab in the Dev UI (#546 )

ssbushi
authored and
tagboola
committed
hide inspect button if no traces (#548 )

ssbushi
authored and
tagboola
committed
[UI] Add typewriter effect to welcoem message (#554 )

tonybaroneee
authored and
tagboola
committed
[UI] Tweak logo kerning (#555 )

tonybaroneee
authored and
tagboola
committed
[UI] UI polish for evaluate page (#553 )

tonybaroneee
authored and
tagboola
committed
[UI] Fix issue in action runner JSON pre-fill (#559 )

tonybaroneee
authored and
tagboola
committed
[UI] Update typewriter animation to move left-to-right (#560 )

tonybaroneee
authored and
tagboola
committed
[UI] Show custom metadata attributes last in span details (#563 )

tonybaroneee
authored and
tagboola
committed
[UI] Polish for eval result details pane (#564 )

tonybaroneee
authored and
tagboola
committed
Add support for text-embeddings (#538 )

huangjeff5
authored and
tagboola
committed
[UI] Update default font to Google Sans (#565 )

tonybaroneee
authored and
tagboola
committed
[UI] Update span attributes styling (#568 )

tonybaroneee
authored and
tagboola
committed
[UI] Update border radius globally (#573 )

tonybaroneee
authored and
tagboola
committed
[UI] Clip model playground message loading bar to card radius (#576 )

tonybaroneee
authored and
tagboola
committed
[UI] Prevent shrinkage of breadcrumb chevron (#577 )

tonybaroneee
authored and
tagboola
committed
[UI] Upgrade angular deps to ^17.3.1 (#587)

tonybaroneee
authored and
tagboola
committed
[UI] Add logo lockup to app bar (#588 )

tonybaroneee
authored and
tagboola
committed
[UI] Fix table not rendering for errored traces (#607)

tonybaroneee
authored and
tagboola
committed
[UI] Render base64-encoded images in span output (#606 )

tonybaroneee
authored and
tagboola
committed
[UI] Update label of expand text button (#608 )

tonybaroneee
authored and
tagboola
committed
[UI] Update lockup with new svg asset (#623 )

tonybaroneee
authored and
tagboola
committed
[Eval bugbash] Update tooltip to definitions, visible on entire chip (#624 )

ssbushi
authored and
tagboola
committed
[Eval bugbash] Show errors as errors in eval UI (#626 )

ssbushi
authored and
tagboola
committed
[Eval bugbash] Only show icon if failed evaluator (#635 )

ssbushi
authored and
tagboola
committed
[UI] Fix trace timing display now that they are millis (#638 )

tonybaroneee
authored and
tagboola
committed
[UI] Fix JSON editor to show up for optional inputs as well (#613 )

shrutip90
authored and
tagboola
committed
Add trace id to model playground when error occurs (#631 )

MaesterChestnut
authored and
tagboola
committed
Display context strings separately instead of a big array (#658 )

shrutip90
authored and
tagboola
committed
[UI]: Update date format to medium (#659 )

shrutip90
authored and
tagboola
committed
Update error tooltip (#665 )

ssbushi
authored and
tagboola
committed
[UI] Tighten up kerning on mat tab labels (#680 )

tonybaroneee
authored and
tagboola
committed
[UI] Allow resizing of .pre-container and json editor (#682)

tonybaroneee
authored and
tagboola
committed
[UI] Add tooltips to temperature and top_p controls (#683 )

tonybaroneee
authored and
tagboola
committed
[UI] Fix JSON sample autofill in retriever playground (#684 )

tonybaroneee
authored and
tagboola
committed
[UI] Improve model playground param labels and add tooltips (#686 )

tonybaroneee
authored and
tagboola
committed
[UI] Fix trace status in table (#687 )

tonybaroneee
authored and
tagboola
committed
[UI] Update model icon to sparks (#688 )

tonybaroneee
authored and
tagboola
committed
[UI] Add action type to runner page title (#690 )

tonybaroneee
authored and
tagboola
committed
[UI] Add title and close button to expand text dialog (#691 )

tonybaroneee
authored and
tagboola
committed
[UI] Remove redundant title from action runner (#692 )

tonybaroneee
authored and
tagboola
committed
Pass thru options to API (#695 )

huangjeff5
authored and
tagboola
committed
Bump ragas to 0.0.6 (#719 )

pavelgj
authored and
tagboola
committed
[UI] Cleanup system prompt styling in model playground (#725 )

tonybaroneee
authored and
tagboola
committed
Update system/message placeholders (#727 )

MichaelDoyle
authored and
tagboola
committed
Update Eval Error handling (#685 )

MaesterChestnut
authored and
tagboola
committed
Clarifying label on button formerly known as "Open in Playground" (#636 )

maxl0rd
authored and
tagboola
committed
[UI] Fix callout content not stretching to fit width (#757 )

tonybaroneee
authored and
tagboola
committed
[UI]: Add metrics table in evals results card (#747 )

shrutip90
authored and
tagboola
committed
[UI] Add support for specifying model version in playground (#760 )

tonybaroneee
authored and
tagboola
committed
[UI] Remove Evaluate tab in top nav bar (#765 )

tonybaroneee
authored and
tagboola
committed
[UI] Use flask icon for Evaluate tab (#772 )

tonybaroneee
authored and
tagboola
committed
[UI] Style updates to eval result details (#790 )

tonybaroneee
authored and
tagboola
committed
[UI] Render eval metric name in error callout consistently (#792 )

tonybaroneee
authored and
tagboola
committed
[UI] Fix span duration display (#797 )

tonybaroneee
authored and
tagboola
committed
Show safety errors in the model runner (#800 )

MichaelDoyle
authored and
tagboola
committed
Rename model playground => runner (#803 )

MichaelDoyle
authored and
tagboola
committed
Rename retriever playground => runner (#805 )

MichaelDoyle
authored and
tagboola
committed
[UI] Adjust metrics table to be full-width (#810 )

tonybaroneee
authored and
tagboola
committed
[UI] Only show eval zero state when loaded (#811 )

tonybaroneee
authored and
tagboola
committed
[UI] Set All traces as default in Inspect view (#812 )

tonybaroneee
authored and
tagboola
committed
[UI] ThemeToggleService unit tests (#816 )

shrutip90
authored and
tagboola
committed
[UI] Make spans deep-linkable in trace + flow details views (#819 )

tonybaroneee
authored and
tagboola
committed
[UI] Update model runner title to use selected model in config (#822 )

tonybaroneee
authored and
tagboola
committed
[UI] Clear out images from data-rendered upon receiving new input (#840 )

tonybaroneee
authored and
tagboola
committed
[UI] Hide append mode for models that do not support multiturn (#847 )

tonybaroneee
authored and
tagboola
committed
[UI] Show banner for unsupported models (#848 )

tonybaroneee
authored and
tagboola
committed
[UI] Reset scroll position of input/output when switching spans (#852 )

tonybaroneee
authored and
tagboola
committed
[UI] Hide "Add message" if model does not support multiturn (#853)

tonybaroneee
authored and
tagboola
committed
Fix missed version 0.5.0-rc.1 (#858 )

pavelgj
authored and
tagboola
committed
[UI] Fix display of system prompt (#860 )

tonybaroneee
authored and
tagboola
committed
[UI] Fix tools icon (#862 )

tonybaroneee
authored and
tagboola
committed
[UI] Prevent stuck browser back when redirecting to first evaluation run (#13 )

tonybaroneee
authored and
tagboola
committed
[UI] Add missing app text color style (#16 )

tonybaroneee
authored and
tagboola
committed
[UI] Apply theme to scrollbars (#20 )

tonybaroneee
authored and
tagboola
committed
[UI] Clarify ID in flows/traces tables (#23 )

tonybaroneee
authored and
tagboola
committed
[UI] Show flow error in trace details view, if applicable (#28 )

tonybaroneee
authored and
tagboola
committed
[UI] Fix eval zero state callout spacing (#24 )

tonybaroneee
authored and
tagboola
committed
Export textEmbedding (#36 )

huangjeff5
authored and
tagboola
committed
[UI] Update README doc with up-to-date instructions (#50 )

tonybaroneee
authored and
tagboola
committed
[UI] Create skeleton prompt runner component (#54 )

tonybaroneee
authored and
tagboola
committed
[UI] Add icon to all view trace buttons (#57 )

tonybaroneee
authored and
tagboola
committed
[UI] Show template in prompt runner next to input (#58 )

tonybaroneee
authored and
tagboola
committed
[UI] Use button toggle group for inspect table filter (#56 )

tonybaroneee
authored and
tagboola
committed
[UI] Update play icon for run/dispatch span states (#60 )

tonybaroneee
authored and
tagboola
committed
More sensible default model params (#65 )

MichaelDoyle
authored and
tagboola
committed
Always clear message when not in chat mode - otherwise if an error is shown, we'll still see the previous message. (#67 )

MichaelDoyle
authored and
tagboola
committed
[UI] Show raw prompt template in modal (#70 )

tonybaroneee
authored and
tagboola
committed
Nesting user input in prompt runner (#72 )

maxl0rd
authored and
tagboola
committed
[UI] Add support for prompt variants (#74 )

tonybaroneee
authored and
tagboola
committed
Allow system role for Gemini 1.5 Pro (#85 )

tonybaroneee
authored and
tagboola
committed
Create modular component for a multi-modal message (#83 )

MichaelDoyle
authored and
tagboola
committed
Update faithfulness to v0.1.7 (#87 )

ssbushi
authored and
tagboola
committed
[UI] Add prompt variant to query params to support deep-linking (#88 )

tonybaroneee
authored and
tagboola
committed
[UI] Fix race condition when setting content in monaco (#96 )

tonybaroneee
authored and
tagboola
committed
[UI] Small visual fix in app nav bar (#98 )

tonybaroneee
authored and
tagboola
committed
[UI] Fix incorrect height for modal runner header (#101 )

tonybaroneee
authored and
tagboola
committed
[UI] Update placeholder label for model version select (#100 )

tonybaroneee
authored and
tagboola
committed
Message list component (#84 )

authored and
tagboola
committed
[UI] Fix view evaluation report button to read correct metdata (#119 )

tonybaroneee
authored and
tagboola
committed
[UI] Save action sidebar expansion state to localStorage (#120 )

tonybaroneee
authored and
tagboola
committed
[UI]: Move model config params to a separate component (#103 )

shrutip90
authored and
tagboola
committed
[UI] Update model runner to use the new model config component (#124 )

shrutip90
authored and
tagboola
committed
[UI] Pull the new defaults for model config into the new config component (#125 )

shrutip90
authored and
tagboola
committed
[UI] Add ability to export prompt file from model runner (#115 )

tonybaroneee
authored and
tagboola
committed
[UI] Fix model versions not being loaded on initial render (#131 )

tonybaroneee
authored and
tagboola
committed
Integrate the new MessageList component into the ModelRunner (#114 )

MichaelDoyle
authored and
tagboola
committed
[UI] Refactor model-config to use signals (#133 )

tonybaroneee
authored and
tagboola
committed
Create placeholder for system prompt and first user message (#144 )

MichaelDoyle
authored and
tagboola
committed
[UI] Remove oops from model config template (#143 )

tonybaroneee
authored and
tagboola
committed
Ensure selected model is set when using left nav (#148 )

MichaelDoyle
authored and
tagboola
committed
[UI] Prevent button icons from flex-shrinking (#151 )

tonybaroneee
authored and
tagboola
committed
Show large multimedia in a modal (#156 )

MichaelDoyle
authored and
tagboola
committed
Enable all image types in model runner (#160 )

MichaelDoyle
authored and
tagboola
committed
Re-enable gemini vision models (#168 )

MichaelDoyle
authored and
tagboola
committed
[UI] Remove system prompt for single-turn models (#169)

shrutip90
authored and
tagboola
committed
Set a reasonable (but arbitrary) number of media files per message (#172)

MichaelDoyle
authored and
tagboola
committed
[UI] Remove obsolete MONACO_PATH provider (unused) (#182 )

tonybaroneee
authored and
tagboola
committed
[UI] Sort eval metrics for consistent/comparable viewing (#209 )

tonybaroneee
authored and
tagboola
committed
change action latency name (#200 )

jba
authored and
tagboola
committed
Add strong response types for vertex evaluators
tagboola
committed