Skip to content

Unable to use CostFrugalTuner in AutoML  #6604

@hieu-n

Description

@hieu-n

System Information (please complete the following information):

  • OS & Version: Ubuntu 22.04.2 LTS
  • ML.NET Version: 2.0.1
  • .NET Version: 7.0.101

Describe the bug
Trying to SetCostFrugalTuner for an AutoML experiment and got the exception:

Unhandled exception. System.AggregateException: One or more errors occurred. (Object reference not set to an instance of an object.)
 ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.ML.AutoML.CostFrugalTuner..ctor(AutoMLExperimentSettings settings, ITrialResultManager trialResultManager)
   at Microsoft.ML.AutoML.AutoMLExperimentExtension.<>c.<SetCostFrugalTuner>b__9_0(IServiceProvider service)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite factoryCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(Type serviceType)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
   at Microsoft.ML.AutoML.AutoMLExperiment.RunAsync(CancellationToken ct)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Microsoft.ML.AutoML.AutoMLExperiment.Run()
   at SampleProject.Program.Main(String[] args) in /home/myhome/SampleProject/Program.cs:line 54  

To Reproduce
Steps to reproduce the behavior:

  1. Create a new console application with the command dotnet new console -o SampleProject.
  2. Adding ML.Net packages.
  3. Create a simple code to use AutoML to train a binary classification model and SetCostFrugalTuner().
  4. Run and encounter the exception.

Expected behavior
The program should run without exceptions and the model is successfully trained.

Screenshots, Code, Sample Projects
SampleProject.csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net7.0</TargetFramework>
    <Nullable>enable</Nullable>
  </PropertyGroup>
  
  <ItemGroup>
    <PackageReference Include="Microsoft.ML" Version="2.0.1" />
    <PackageReference Include="Microsoft.ML.AutoML" Version="0.20.1" />
    <PackageReference Include="Microsoft.ML.LightGbm" Version="2.0.1" />
    <PackageReference Include="Microsoft.ML.Probabilistic.Compiler" Version="0.4.2301.301" />
    <PackageReference Include="MKL.NET.linux-x64" Version="2022.0.1.117" />
  </ItemGroup>

</Project>

Program.cs

using Microsoft.ML;
using Microsoft.ML.AutoML;
using Microsoft.ML.Data;

namespace SampleProject;

public class HousingData
{
    public float Size { get; set; }

    [VectorType(3)]
    public float[] HistoricalPrices { get; set; }

    public float CurrentPrice { get; set; }
    public bool Label { get; set; }
}

public class Program
{
    public static void Main(string[] args)
    {
        HousingData[] inMemoryCollection = new HousingData[]
        {
            new HousingData
            {
                Size = 700f,
                HistoricalPrices = new float[]
                {
                    100000f, 3000000f, 250000f
                },
                CurrentPrice = 500000f,
                Label = true
            },
            new HousingData
            {
                Size = 1000f,
                HistoricalPrices = new float[]
                {
                    600000f, 400000f, 650000f
                },
                CurrentPrice = 700000f,
                Label = false
            }
        };
        // Create MLContext
        MLContext mlContext = new MLContext();

        //Load Data
        IDataView data = mlContext.Data.LoadFromEnumerable(inMemoryCollection);
        var pipeline = mlContext.Auto().Featurizer(data, excludeColumns: new[]{"Label"}).Append(
            mlContext.Auto().BinaryClassification());
        var experiment = new MLContext().Auto().CreateExperiment();
        experiment.SetPipeline(pipeline).SetTrainingTimeInSeconds(60).SetDataset(data).SetCostFrugalTuner();
        experiment.Run();
    }
}

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions