Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ArgumentOutOfRangeException when running Microsoft.ML.Fairlearn.Tests.GridSearchTest.TestGridSearchTrialRunner2 #6842

Open
ericstj opened this issue Sep 27, 2023 · 0 comments
Labels
blocking-clean-ci Blocking PR or rolling builds Known Build Error Use this to report build issues in the .NET Helix tab untriaged New issue has not been triaged

Comments

@ericstj
Copy link
Member

ericstj commented Sep 27, 2023

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=419700
Build error leg or test failing: Microsoft.ML.Fairlearn.Tests.WorkItemExecution
Pull request: #6837

Error Message

        [Source=AutoMLExperiment, Kind=Trace] Channel started
        [Source=AutoMLExperiment, Kind=Info] Update Running Trial - Id: 0
        [Source=AutoMLExperiment, Kind=Trace] Microsoft.ML.AutoML.DefaultPerformanceMonitor has been started
        [Source=AutoMLExperiment, Kind=Trace] current CPU: 0, current Memory(mb): 48.51171875
        [Source=AutoMLExperiment, Kind=Trace] maximum memory usage: , PeakMemoryUsage: 48.51171875 trialIsCancelled: False
        [Source=AutoMLExperiment, Kind=Trace] trial setting - {"TrialId":0,"StartedAtUtc":"2023-09-26T20:49:48.6702343Z","EndedAtUtc":null,"Parameter":{"_lambda_search_space":{"2_neg":-4.00442839,"2_pos":0.776618361,"3_neg":-2.88146996,"3_pos":8.18861198},"_pipeline_":{"_SCHEMA_":"e0 * e5","e0":{},"e1":{"NumberOfLeaves":1487,"MinimumExampleCountPerLeaf":23,"NumberOfTrees":61,"MaximumBinCountPerFeature":114,"FeatureFraction":0.4980983951658306,"LearningRate":1.1821894183157802E-07,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight","DiskTranspose":false},"e2":{"NumberOfTrees":5,"NumberOfLeaves":41,"FeatureFraction":0.404329568,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e3":{"NumberOfLeaves":17267,"MinimumExampleCountPerLeaf":63,"LearningRate":5.8468910203332519E-09,"NumberOfTrees":2221,"SubsampleFraction":2.8959196043297387E-08,"MaximumBinCountPerFeature":11,"FeatureFraction":0.88016485093703201,"L1Regularization":4.0092348571131609E-05,"L2Regularization":0.093939385599205991,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e4":{"L1Regularization":9.83808041,"L2Regularization":4.45183802,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e5":{"L1Regularization":80.4055481,"L2Regularization":2602.4187,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"}}}}
        [Source=AutoMLExperiment, Kind=Trace] current CPU: 0.09375, current Memory(mb): 72.10546875
        [Source=AutoMLExperiment, Kind=Trace] maximum memory usage: , PeakMemoryUsage: 72.10546875 trialIsCancelled: False
        [Source=AutoMLExperiment, Kind=Trace] 
        Exception thrown during Trial 0 with configuration {"TrialId":0,"StartedAtUtc":"2023-09-26T20:49:48.6702343Z","EndedAtUtc":null,"Parameter":{"_lambda_search_space":{"2_neg":-4.00442839,"2_pos":0.776618361,"3_neg":-2.88146996,"3_pos":8.18861198},"_pipeline_":{"_SCHEMA_":"e0 * e5","e0":{},"e1":{"NumberOfLeaves":1487,"MinimumExampleCountPerLeaf":23,"NumberOfTrees":61,"MaximumBinCountPerFeature":114,"FeatureFraction":0.4980983951658306,"LearningRate":1.1821894183157802E-07,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight","DiskTranspose":false},"e2":{"NumberOfTrees":5,"NumberOfLeaves":41,"FeatureFraction":0.404329568,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e3":{"NumberOfLeaves":17267,"MinimumExampleCountPerLeaf":63,"LearningRate":5.8468910203332519E-09,"NumberOfTrees":2221,"SubsampleFraction":2.8959196043297387E-08,"MaximumBinCountPerFeature":11,"FeatureFraction":0.88016485093703201,"L1Regularization":4.0092348571131609E-05,"L2Regularization":0.093939385599205991,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e4":{"L1Regularization":9.83808041,"L2Regularization":4.45183802,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"},"e5":{"L1Regularization":80.4055481,"L2Regularization":2602.4187,"LabelColumnName":"y","FeatureColumnName":"Features","ExampleWeightColumnName":"signedWeight"}}}}
        
        Exception Details: AUC is not defined when there is no negative class in the data
        Parameter name: NegSample
        
        Abandoning Trial 0 and continue training.
        
        [Source=AutoMLExperiment, Kind=Trace] AUC is not defined when there is no negative class in the data
        Parameter name: NegSample   at Microsoft.ML.Data.EvaluatorBase`1.AucAggregatorBase`1.ComputeWeightedAuc(Double& unweighted) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\AucAggregator.cs:line 116
           at Microsoft.ML.Data.BinaryClassifierEvaluator.Aggregator.Finish() in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\BinaryClassifierEvaluator.cs:line 705
           at Microsoft.ML.Data.BinaryClassifierEvaluator.<>c__DisplayClass32_0.<GetAggregatorConsolidationFuncs>b__0(UInt32 stratColKey, ReadOnlyMemory`1 stratColVal, Aggregator agg) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\BinaryClassifierEvaluator.cs:line 261
           at Microsoft.ML.Data.EvaluatorBase`1.ProcessData(IDataView data, RoleMappedSchema schema, Func`2 activeColsIndices, TAgg aggregator, AggregatorDictionaryBase[] dictionaries) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\EvaluatorBase.cs:line 183
           at Microsoft.ML.Data.EvaluatorBase`1.Microsoft.ML.Data.IEvaluator.Evaluate(RoleMappedData data) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\EvaluatorBase.cs:line 33
           at Microsoft.ML.Data.BinaryClassifierEvaluator.Evaluate(IDataView data, String label, String score, String predictedLabel) in D:\a\_work\1\s\src\Microsoft.ML.Data\Evaluators\BinaryClassifierEvaluator.cs:line 951
           at Microsoft.ML.Fairlearn.GridSearchTrailRunner.RunAsync(TrialSettings settings, CancellationToken ct) in D:\a\_work\1\s\src\Microsoft.ML.Fairlearn\Reductions\GridSearchTrialRunner.cs:line 89
           at Microsoft.ML.AutoML.AutoMLExperiment.<RunAsync>d__24.MoveNext() in D:\a\_work\1\s\src\Microsoft.ML.AutoML\AutoMLExperiment\AutoMLExperiment.cs:line 273
        [Source=AutoMLExperiment, Kind=Info] Update Failed Trial - Id: 0 - Pipeline: Unknown=>SdcaLogisticRegressionBinary

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "System.ArgumentOutOfRangeException : AUC is not defined when there is no negative class in the data",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": true
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=419700
Error message validated: System.ArgumentOutOfRangeException : AUC is not defined when there is no negative class in the data
Result validation: ❌ Known issue did not match with the provided build.
Validation performed at: 9/27/2023 6:13:57 PM UTC

Report

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 0
@ghost ghost added the untriaged New issue has not been triaged label Sep 27, 2023
@ericstj ericstj added blocking-clean-ci Blocking PR or rolling builds Known Build Error Use this to report build issues in the .NET Helix tab labels Sep 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocking-clean-ci Blocking PR or rolling builds Known Build Error Use this to report build issues in the .NET Helix tab untriaged New issue has not been triaged
Projects
None yet
Development

No branches or pull requests

1 participant