Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement NLP transforms, KS alarm + detector #163

Open
wants to merge 29 commits into
base: dev
Choose a base branch
from
Open

Implement NLP transforms, KS alarm + detector #163

wants to merge 29 commits into from

Conversation

Anmol-Srivastava
Copy link
Contributor

@Anmol-Srivastava Anmol-Srivastava commented Oct 17, 2023 •
edited
Loading

As experimental code, this serves as an outline of the NLP framework rather than final versions. Closes #152

Added

  • transforms (Representation removed) related to alibi-detect example
  • basic Detector and KS test Alarm
  • updated wilds_datasets.ipynb example to reflect testing on civil-comments dataset

Outstanding Issues

  • design of the KS Alarm scheme -- test occurs feature-wise, but how should drift alarm?

@Anmol-Srivastava Anmol-Srivastava self-assigned this Oct 17, 2023
@Anmol-Srivastava Anmol-Srivastava linked an issue Oct 17, 2023 that may be closed by this pull request
Design nlp submodule and skeleton of drift detection methods #152
Open
@tms-bananaquit
Copy link
Collaborator

Will need to add citations and update references for alibi-detect's paper and/or documentation before merging the experimental branch.

@Anmol-Srivastava
Copy link
Contributor Author

This is essentially ready to be merged in -- citations are added and the code is stable enough.

This also now closes #153 #154 and can enable proper integration into menelaus

@Anmol-Srivastava
Copy link
Contributor Author

Now also closes #164

tms-bananaquit
tms-bananaquit requested changes Nov 9, 2023
View reviewed changes
docs/source/examples/nlp/wilds_datasets.ipynb Outdated
Copy link
Collaborator

@tms-bananaquit tms-bananaquit Nov 9, 2023 •
edited by Anmol-Srivastava
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Should probably remove output (warnings) that references local files (e.g. c:\Users\ASRIVASTAVA)
  • Should test building on readthedocs to make sure this renders correctly

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolving once I've done these

menelaus/experimental/alarm.py Show resolved Hide resolved
setup.cfg Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tms-bananaquit tms-bananaquit tms-bananaquit requested changes

Requested changes must be addressed to merge this pull request.

Assignees

@Anmol-Srivastava Anmol-Srivastava

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Design nlp submodule and skeleton of drift detection methods
2 participants
@Anmol-Srivastava @tms-bananaquit