Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Propagate span annotation metadata to examples on all mutations #4195

Merged
merged 5 commits into from
Aug 13, 2024

Conversation

anticorrelator
Copy link
Contributor

Resolves #4167

Span annotations should be propagated to dataset example metadata when added. However this functionality wasn't available on all mutations that created dataset examples.

This PR ensures that even if creating a dataset example via the add_examples_to_dataset mutation, associated span annotations are added to the metadata.

@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Aug 9, 2024
@mikeldking
Copy link
Contributor

@anticorrelator I think you fixed the wrong code path. If you test end-to-end I think you would see that you need to modify

def as_example_revision(self) -> SpanAsExampleRevision:

where the UI materializes the span as a dataset example before performing the create Mutation. Can you validate e2e that this PR works?

@anticorrelator
Copy link
Contributor Author

@anticorrelator I think you fixed the wrong code path. If you test end-to-end I think you would see that you need to modify

def as_example_revision(self) -> SpanAsExampleRevision:

where the UI materializes the span as a dataset example before performing the create Mutation. Can you validate e2e that this PR works?

Sounds good, will verify. I think this might still work though, just with an incomplete preview. When we try to add the Example revision it'll then propagate the span annotations on insert as long as the span_id field is populated.

@anticorrelator anticorrelator merged commit 181e021 into main Aug 13, 2024
11 checks passed
@anticorrelator anticorrelator deleted the dustin/add-annotation-metadata-on-all-mutations branch August 13, 2024 03:04
fjcasti1 added a commit that referenced this pull request Aug 16, 2024
* main: (58 commits)
  Update OpenAPI spec (#4260)
  Default to adding span annotations to queue (#4259)
  docs: Add human feedback notebook tutorial (#4257)
  refactor: daemon task abstraction (#4255)
  feat(auth): create system key (#4235)
  chore: add vision fixture (#4250)
  docs: fix variable name typo in run experiments doc (#4249)
  build(deps): bump certifi (#4147)
  chore(main): release arize-phoenix 4.24.0 (#4225)
  chore(main): release arize-phoenix-evals 0.15.0 (#3920)
  feat(experiments): ability to specify concurrency in run_experiment and evaluate_experiment (#4189)
  ci(evals): limit `mistralai` below version 1.0 (#4241)
  feat(auth): user / system api key resolvers (#4232)
  feat(auth): add user role, exclude system in user lists (#4229)
  docs: Add multimodal image reasoning tutorial with llama index (#4210)
  chore(main): release arize-phoenix 4.23.0 (#4216)
  docs: No subject (GITBOOK-818)
  build(deps): bump streamlit (#4214)
  fix: Propagate span annotation metadata to examples on all mutations (#4195)
  feat(auth): users table in settings (#4221)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] annotations not propagating to dataset metadata
2 participants