Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Preserve the output of the eval notebooks #1559

Closed
mikeldking opened this issue Oct 4, 2023 · 8 comments · Fixed by #1590 or #1625
Closed

CI: Preserve the output of the eval notebooks #1559

mikeldking opened this issue Oct 4, 2023 · 8 comments · Fixed by #1590 or #1625
Assignees
Labels
documentation Improvements or additions to documentation good first issue Good for newcomers

Comments

@mikeldking
Copy link
Contributor

mikeldking commented Oct 4, 2023

We would like to have eval notebooks that contain a clean output of the results without containing secret openAI keys in the metadata. The notebooks need to be a bit cleaner so it's not polluted with output.

Acceptance criteria

  • notebook metadata cleared
  • OpenAI keys etc are not embedded
  • cleanup the notebooks to not output massive progress bars etc.
@mikeldking mikeldking added documentation Improvements or additions to documentation good first issue Good for newcomers labels Oct 4, 2023
@pbadhe
Copy link
Collaborator

pbadhe commented Oct 6, 2023

If no one is working yet, I can try this. It's just all the existing notebooks from tutorials/evals/* which are required to preserve output in this PR, right?

@axiomofjoy
Copy link
Contributor

axiomofjoy commented Oct 6, 2023

@pbadhe That would be great! Let me know if you want to chat on it. Feel free to assign the ticket to yourself.

@axiomofjoy
Copy link
Contributor

axiomofjoy commented Oct 6, 2023

It's just all the existing notebooks from tutorials/evals/* which are required to preserve output in this PR, right?

Exactly. We want to continue to scrub notebook metadata, but keep the output. And maybe do a sanity check to make sure our API keys stored in getpass aren't being saved in the output somehow.

@pbadhe
Copy link
Collaborator

pbadhe commented Oct 11, 2023

Thanks for assigning @mikeldking. @axiomofjoy I think I cannot change the assignee that's why didn't assign the issue to myself.

@mikeldking
Copy link
Contributor Author

The pre-commit still contains the wrong sanitization logic so I will re-open https://github.com/Arize-ai/phoenix/blob/main/.pre-commit-config.yaml

@mikeldking mikeldking reopened this Oct 13, 2023
@pbadhe
Copy link
Collaborator

pbadhe commented Oct 13, 2023

Sorry, missed it. @mikeldking.

@mikeldking
Copy link
Contributor Author

Sorry, missed it. @mikeldking.

No worries @pbadhe !

@axiomofjoy
Copy link
Contributor

Good catch @mikeldking

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation good first issue Good for newcomers
Projects
Archived in project
3 participants