Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a New WordNoMatch Descriptor to Evidently #1309

Closed
elenasamuylova opened this issue Sep 23, 2024 · 1 comment
Closed

Add a New WordNoMatch Descriptor to Evidently #1309

elenasamuylova opened this issue Sep 23, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request hacktoberfest Accepted contributions will count towards your hacktoberfest PRs

Comments

@elenasamuylova
Copy link
Collaborator

elenasamuylova commented Sep 23, 2024

Add a New WordNoMatch Descriptor to Evidently

About Hacktoberfest contributions: https://github.com/evidentlyai/evidently/wiki/Hacktoberfest-2024

Description:

Evidently already has an ExcludesWords() descriptor that checks if the text does not contain any specified words from a shared list.

However, sometimes you might need to check that the text does not contain words specific to each row instead of a shared list.

Example:

Question Response Forbidden Words
"Can I cancel my subscription at any time?" "You are allowed to cancel at any time, and we guarantee that you will receive a refund." ["guarantee", "allowed", "refund"]

What to Implement:

The new WordNoMatch() descriptor should:

  1. Accept a with_column parameter: This column contains a list of forbidden words specific to each row.
  2. Accept a lemmatize parameter (default True): When True, this will consider inflected or variant forms of words. Works the same as in the ExcludesWords() descriptor.
  3. Return True/False:
    • Return True if the words from the list are not present in the row's text.
    • Return False if forbidden words are present.

References:

  • Check the ExcludesWords() descriptor as reference.
  • For a two-column descriptor implementation, check the SemanticSimilarity descriptor and the CustomPairColumnEval template.
@elenasamuylova elenasamuylova added enhancement New feature or request hacktoberfest Accepted contributions will count towards your hacktoberfest PRs labels Sep 23, 2024
@jon-bown
Copy link
Contributor

jon-bown commented Oct 6, 2024

Opened a joint PR for this issue and #1308

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request hacktoberfest Accepted contributions will count towards your hacktoberfest PRs
Projects
None yet
Development

No branches or pull requests

3 participants