Assemblies loaded by the Default LoadContext do not properly evaluate function.deps.json

[brettsam]

It appears that when the host starts up and there's a Durable Functions retry that fires, something in the way Json.NET loads assemblies forces the assembly to load in the Default LoadContext, rather than in the Shared LoadContext. On this path, the function.deps.json file isn't consulted and native assemblies do not resolve correctly.

[brettsam]

@brendonatkins -- I was able to workaround the issue from your repro by removing Microsoft.Data.SqlClient.dll from my bin directory, which forced it to load the correct version at runtime. Give this a try in your csproj file and see if it gets things working for you until the fix is deployed:

  <Target Name="DeleteSqlAfterBuild" AfterTargets="Build">
    <Delete Files="$(TargetDir)bin/Microsoft.Data.SqlClient.dll" />  
  </Target>
    
  <Target Name="DeleteSqlAfterPublish" AfterTargets="Publish">    
    <Delete Files="$(PublishDir)bin/Microsoft.Data.SqlClient.dll" />  
  </Target>

[IanKemp]

@brettsam @fabiocav I have a feeling this change will also fix the issue I'm having in #5950. Please can you get a beta build of azure-functions-core-tools, including this fix, out ASAP? I tried making a build myself but there is no documentation on how to do so and this issue has already taken up too much of my time.

[brettsam]

@IanKemp -- I was able to use your repro solution and verified that this does fix the issue. I now get Format of the initialization string does not conform to specification starting at index 0. rather than PlatformNotSupported.

We typically don't publish anything new for Core Tools until the release is in production, otherwise you get a mismatch of what you run locally vs what is deployed. That being said, if you want to validate the fix, I should be able to put something together with the caveat that deploying it would still have the old, unfixed, code.

[IanKemp]

@IanKemp -- I was able to use your repro solution and verified that this does fix the issue. I now get Format of the initialization string does not conform to specification starting at index 0. rather than PlatformNotSupported.

We typically don't publish anything new for Core Tools until the release is in production, otherwise you get a mismatch of what you run locally vs what is deployed. That being said, if you want to validate the fix, I should be able to put something together with the caveat that deploying it would still have the old, unfixed, code.

I'd be happy to "beta test" the fix in my development environment. Not only would it give me some closure (I've been banging my head against this particular wall for the last day-and-a-half), it will also confirm that the fix does actually work for my particular scenario. (It will also reassure my boss that this isn't a show-stopper preventing us from using Azure Functions.)

That said, obviously I'd like to deploy this code to real Azure sometime in the near future - are you able to provide an estimate of when this fix might be rolled out? I'm not asking for a hard date; something like a week, a month, etc. would be more than sufficient (and I won't hold you to it).

[fabiocav]

@IanKemp for an ETA (again, not committing to a date), I would expect to see this fix rolling to the hosted environment early/mid next week. Once the deployment starts, it takes about a week to complete.
If you subscribe to https://github.com/Azure/app-service-announcements/issues, you'll be notified when the release starts.

[IanKemp]

@IanKemp for an ETA (again, not committing to a date), I would expect to see this fix rolling to the hosted environment early/mid next week. Once the deployment starts, it takes about a week to complete.
If you subscribe to https://github.com/Azure/app-service-announcements/issues, you'll be notified when the release starts.

Thanks!

Will a new version of the Azure Functions Core Tools be available around the same time?

[APIWT]

@brettsam @fabiocav Thank you so much for the suggested work around and open communication about the fix being deployed to production. Quick question: If I apply the work around suggested in my .csproj (or a Directory.Build.targets, or similar), will I have any issues once the fix is deployed to Azure? In other words, will I have to quickly deploy a new version that removes the work around? Or will it continue to work with this work around without any changes?

[APIWT]

@brettsam @fabiocav Sorry to be a pest, but just a bump on the comment above. We are planning on shipping a new durable function to production very soon and I want to be able to have confidence that this workaround should be safe to use beyond the short term.

[brettsam]

Sorry I missed your comment! You will be fine with the workaround when the fix rolls out -- the same native assembly will be loaded.

[IanKemp]

Any update on new version of the Core Tools?

[IanKemp]

@brettsam The workaround you supplied isn't going to do anything since the Delete task's Files parameter requires an array of files to be passed to it, not a glob. (Yay MSBuild being old and crusty!) The following is tested to work for anyone using either version of the SqlClient assemblies:

  <Target Name="DeleteSqlClientAfterBuild" AfterTargets="Build">
    <ItemGroup>
      <BuildSqlClientBinaries Include="$(TargetDir)bin/*.Data.SqlClient.dll"/>
    </ItemGroup>
    <Delete Files="@(BuildSqlClientBinaries)" />
  </Target>
  <Target Name="DeleteSqlClientAfterPublish" AfterTargets="Publish">
    <ItemGroup>
      <PublishSqlClientBinaries Include="$(PublishDir)bin/*.Data.SqlClient.dll"/>
    </ItemGroup>
    <Delete Files="@(PublishSqlClientBinaries)" />
  </Target>

However, this doesn't fix the System.Data.SqlClient issue on Azure, as I now get the FileNotFoundException instead.

[IanKemp]

@brettsam @fabiocav This fix still isn't present in the latest release of the Host Runtime (3.0.13901 - confirmed by inspecting the artifacts attached to that release) despite it being merged 2 weeks ago. This means that the latest version of the Host Runtime being deployed lacks this fix.

Please advise, what is the hold-up? It almost seems as if the code being built, and therefore deployed, is not the code from this repo, or not from the dev branch.

[IanKemp]

Excellent, seems like this is finally in 3.0.13904 - but when that will roll out to Azure is anyone's guess.

The Core Tools will need updating, as well as Microsoft.NET.Sdk.Functions.

[tbertenshaw]

Is it possible to evaluate these fixes when debugging locally to see if it resolves my issue with

Could not load file or assembly 'System.Data.SqlClient, Version=4.6.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Could not find or load a specific file.

I have these versions

<PackageReference Include="Microsoft.Azure.Functions.Extensions" Version="1.0.0" />
<PackageReference Include="Microsoft.NET.Sdk.Functions" Version="3.0.9" />