Implement GLEU and F1 NLP evaluators #6555

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

peterwald merged 6 commits into dotnet:main from peterwald:nlp

Jul 1, 2025

Member

peterwald commented Jun 30, 2025 •

edited by dotnet-policy-service bot

Loading

Microsoft Reviewers: Open in CodeFlow


          Add GLEU and F1 evaluators.

c8ab5b9

peterwald self-assigned this

peterwald requested a review from a team as a code owner

June 30, 2025 14:46

peterwald added the area-ai-eval label

peterwald mentioned this pull request

Implement GLEU, F1 score evaluation for NLP evaluations #6545

Closed

peterwald added 2 commits

June 30, 2025 10:50


          Update READMEs

ba40e5e


          Use arrays in place of IEnumerable for internal processing.

f5670be

peterwald mentioned this pull request

Efficiency updates to BLEU score processing #6556

Closed


          Merge changes from other PR

aaa24e1

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/BLEUEvaluator.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/BLEUEvaluator.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/BLEUEvaluatorContext.cs Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/BLEUEvaluatorContext.cs Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/BLEUEvaluatorContext.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/Common/BLEUAlgorithm.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/Common/BLEUAlgorithm.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/Common/BLEUAlgorithm.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/Common/MatchCounter.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/Common/NGramExtensions.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/Common/BLEUAlgorithm.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/Common/NGramExtensions.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/Common/NGramExtensions.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/Common/F1Algorithm.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/Common/GLEUAlgorithm.cs Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/Common/F1Algorithm.cs Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/Common/GLEUAlgorithm.cs Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/Common/GLEUAlgorithm.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/F1Evaluator.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/F1Evaluator.cs Outdated Show resolved Hide resolved

shyamnamboodiripad reviewed

View reviewed changes

src/Libraries/Microsoft.Extensions.AI.Evaluation.NLP/F1Evaluator.cs Outdated Show resolved Hide resolved

This was referenced Jul 30, 2025

Bump the dotnet group with 4 updates OPS-E2E-PPE/docs#3735

Closed

Bump the dotnet group with 3 updates OPS-E2E-PPE/docs#3738

Closed

Bump the dotnet group with 2 updates OPS-E2E-PPE/docs#3742

Closed

Bump the dotnet group with 3 updates OPS-E2E-PPE/docs#3747

Closed

Bump Microsoft.Extensions.AI and System.Threading.RateLimiting OPS-E2E-PPE/docs#3754

Closed

Bump Azure.Identity and 14 others Back-Buddy/Backend#96

Closed

Bump Microsoft.Extensions.Http.Resilience and Microsoft.Extensions.ServiceDiscovery InfinityFlowApp/aspire-temporal#180

Closed

Bump Microsoft.Extensions.AI and 2 others devlooped/AI.Benchmarks#38

Closed

Bump the dotnet group with 2 updates CamSoper/dotnet.docs#1797

Closed

Bump the dotnet group with 2 updates CamSoper/dotnet.docs#1846

Closed

Bump the dotnet group with 2 updates CamSoper/dotnet.docs#1855

Closed

Bump the dotnet group with 3 updates CamSoper/dotnet.docs#1858

Closed

Bump the dotnet group with 2 updates CamSoper/dotnet.docs#1862

Closed

Bump the dotnet group with 4 updates CamSoper/dotnet.docs#1863

Closed

Bump the dotnet group with 2 updates CamSoper/dotnet.docs#1868

Closed

Bump the dotnet group with 2 updates CamSoper/dotnet.docs#1872

Closed

Bump the dotnet group with 2 updates gewarren/docs#3668

Closed

Bump the dotnet group with 2 updates gewarren/docs#3672

Closed

Bump the dotnet group with 2 updates gewarren/docs#3703

Closed

Bump Microsoft.Extensions.AI and System.Threading.RateLimiting gewarren/docs#3732

Closed

Bump DiffEngine and 10 others StarsStrucke/CockroachDB#1

Closed

Bump Microsoft.Extensions.Http.Resilience to 9.7.0 dotnet/aspire-samples#928

Closed

Bump Microsoft.Extensions.Http.Resilience from 9.4.0 to 9.7.0 dotnet/aspire-samples#931

Closed

Bump Microsoft.Extensions.Http.Resilience to 9.7.0 dotnet/aspire-samples#932

Closed

Bump Microsoft.Extensions.Http.Resilience from 9.4.0 to 9.7.0 dotnet/aspire-samples#933

Closed

Bump Microsoft.Extensions.Http.Resilience to 9.7.0 dotnet/aspire-samples#934

Closed

Bump the patch group with 16 updates DEFRA/pha-import-notifications#216

Closed

Bump the patch group with 17 updates DEFRA/trade-imports-decision-comparer#127

Closed

Bump Aspire.Hosting.AppHost and 9 others omerkocadere/MJAspire#1

Closed

github-actions bot locked and limited conversation to collaborators

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels