Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

This commit implements the F-beta score metric #1543

Merged

Commits on Oct 20, 2024

  1. This commit implements the F-beta score metric

    for the AnswerCorrectness class.
    Yuri-Albuquerque committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    e1f0ca8 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2024

  1. also implements the F-beta score for

    _factual_correctness, which is a
    weighted harmonic mean of precision and recall,
    where the recall is weighted by a factor of beta.
    The F-beta score is defined as:
    F-beta = (1 + beta^2) * (precision * recall) / (beta^2 * precision + recall)
    The F-beta score is a generalization of the F1 score,
    where beta = 1.0. The F1 score is the harmonic mean of
    precision and recall, and is defined as:
    F1 = 2 * (precision * recall) / (precision + recall)
    Yuri-Albuquerque committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    1c339e9 View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. returning to the original recall and precision

    calculation in factual correctness and keeping
    the f1 score as f1-beta score as requested.
    Yuri-Albuquerque committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    ed3b3f3 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2024

  1. remove ALL_METRICS

    shahules786 committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    92166c0 View commit details
    Browse the repository at this point in the history
  2. add fbeta score

    shahules786 committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    69cfa99 View commit details
    Browse the repository at this point in the history
  3. replace by fbeta score

    shahules786 committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    a0f6917 View commit details
    Browse the repository at this point in the history
  4. removed unused imports

    shahules786 committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    c432b19 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    8d5050c View commit details
    Browse the repository at this point in the history