Java Azure Functions - The function runtime is unable to start - Could not load type 'Microsoft.Azure.ServiceBus.Management.ManagementClient

[hci2]

I deployed a new version of my Azure Functions based on Java. Now there is following error toast:

Error:
The function runtime is unable to start. Microsoft.Azure.WebJobs.ServiceBus: Could not load type 'Microsoft.Azure.ServiceBus.Management.ManagementClient' from assembly 'Microsoft.Azure.ServiceBus, Version=3.0.2.0, Culture=neutral, PublicKeyToken=7e34167dcc6d6d8c'.
Session Id: a9f3c42dbb524ceca2c5158df9460c97
Timestamp: 2019-12-02T18:21:46.680Z

It sounds for me a bit like that: Common errors when upgrading to 2.0.12050 (or newer)

The interesting thing is that I changed nothing. Is the Function App Version automatically upgraded?

I also checked the logs via kudu and log stream (application logs).

logs:

2019-12-02T11:08:04.412 [Information] Starting JobHost
2019-12-02T11:08:04.415 [Information] Starting Host (HostId=functionapp-dtw-worker-dp-master, InstanceId=aaab65a2-acef-4e4c-a8be-c3a1b13a795c, Version=2.0.12888.0, ProcessId=6860, AppDomainId=1, InDebugMode=True, InDiagnosticMode=False, FunctionsExtensionVersion=~2)
2019-12-02T11:08:04.467 [Information] Loading functions metadata
2019-12-02T11:08:04.495 [Information] 3 functions loaded
2019-12-02T11:08:04.812 [Information] Generating 3 job function(s)
2019-12-02T11:08:05.126 [Information] Found the following functions:
Host.Functions.lastWillCronJobProcessor
Host.Functions.serviceBusDeadLetterProcessor
Host.Functions.serviceBusProcessor
2019-12-02T11:08:05.351 [Error] A host error has occurred during startup operation '7c3d76bc-730c-401a-86cb-27d04231d515'.
System.TypeLoadException : Could not load type 'Microsoft.Azure.ServiceBus.Management.ManagementClient' from assembly 'Microsoft.Azure.ServiceBus, Version=3.0.2.0, Culture=neutral, PublicKeyToken=7e34167dcc6d6d8c'.
   at Microsoft.Azure.WebJobs.ServiceBus.Listeners.ServiceBusScaleMonitor..ctor(String functionId,EntityType entityType,String entityPath,String connectionString,Lazy`1 receiver,ILoggerFactory loggerFactory)
   at Microsoft.Azure.WebJobs.ServiceBus.Listeners.ServiceBusListener.<.ctor>b__18_0()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication,Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at Microsoft.Azure.WebJobs.ServiceBus.Listeners.ServiceBusListener.GetMonitor()
   at Microsoft.Azure.WebJobs.Host.Listeners.HostListenerFactory.RegisterScaleMonitor(IListener listener,IScaleMonitorManager monitorManager) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Listeners\HostListenerFactory.cs : 107
   at async Microsoft.Azure.WebJobs.Host.Listeners.HostListenerFactory.CreateAsync(CancellationToken cancellationToken) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Listeners\HostListenerFactory.cs : 69
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Host.Listeners.ListenerFactoryListener.StartAsyncCore(CancellationToken cancellationToken) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Listeners\ListenerFactoryListener.cs : 45
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Host.Listeners.ShutdownListener.StartAsync(CancellationToken cancellationToken) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Listeners\ShutdownListener.cs : 29
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.JobHost.StartAsyncCore(CancellationToken cancellationToken) at C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\JobHost.cs : 101
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Script.ScriptHost.StartAsyncCore(CancellationToken cancellationToken) at C:\azure-webjobs-sdk-script\src\WebJobs.Script\Host\ScriptHost.cs : 249
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.Azure.WebJobs.Script.WebHost.WebJobsScriptHostService.UnsynchronizedStartHostAsync(ScriptHostStartupOperation activeOperation,Int32 attemptCount,JobHostStartupMode startupMode) at C:\azure-webjobs-sdk-script\src\WebJobs.Script.WebHost\WebJobsScriptHostService.cs : 235

Log streaming:

Same logs as in the log file via kudu

Do you know why it stopped working? I have no plan to be honest. Did any configuration for the service bus trigger changes?

Check for a solution in the Azure portal

For issues in production, please check for a solution to common issues in the Azure portal before opening a bug. In the Azure portal, navigate to your function app => Platform features => Diagnose and solve problems and the relevant dashboards before opening your issue.

Investigative information

Please provide the following:

• Timestamp: 2019-12-02T18:21:46.680Z
• Function App version (1.0 or 2.0): 2.0.12888.0 (~2)
• Function App name: functionapp-dtw-worker-dp-master
• Application Setting:
  • FUNCTIONS_WORKER_RUNTIME: java
• Function name(s) (as appropriate): serviceBusProcessor
• Session Id: a9f3c42dbb524ceca2c5158df9460c97
• Region: West Europe

Related information

Provide any related information

• Programming language used: Java with Spring Boot, Spring Cloud Functions
• Links to source
• Bindings used: ServiceBusTrigger, TimeTrigger

SUPPLEMENT: I updated the local dependencies like .NET Core Version to the newest v2.2.8 from here: https://dotnet.microsoft.com/download/dotnet-core/2.2

The update does not change the error message. I get the same error exception when I locally start the function with the maven plugin "azure-functions-maven-plugin" version 1.3.4, as you can see below:

[12/2/2019 10:44:26 PM] Starting JobHost
[12/2/2019 10:44:26 PM] Starting Host (HostId=n900b368-1683948780, InstanceId=889e113f-9129-4a24-8fbf-9b560addd68e, Version=2.0.12888.0, ProcessId=2368, AppDomainId=1, InDebugMode=False, InDiagnosticMode=False, FunctionsExtensionVersion=(null))
[12/2/2019 10:44:26 PM] Loading functions metadata
[12/2/2019 10:44:26 PM] 3 functions loaded
[12/2/2019 10:44:26 PM] Generating 3 job function(s)
[12/2/2019 10:44:26 PM] Found the following functions:
[12/2/2019 10:44:26 PM] Host.Functions.lastWillCronJobProcessor
[12/2/2019 10:44:26 PM] Host.Functions.serviceBusDeadLetterProcessor
[12/2/2019 10:44:26 PM] Host.Functions.serviceBusProcessor
[12/2/2019 10:44:26 PM] 
[12/2/2019 10:44:26 PM] A host error has occurred during startup operation '03187631-f70a-45f5-9c3e-008568c4c6a4'.
[12/2/2019 10:44:26 PM] Microsoft.Azure.WebJobs.ServiceBus: Could not load type 'Microsoft.Azure.ServiceBus.Management.ManagementClient' from assembly 'Microsoft.Azure.ServiceBus, Version=3.0.2.0, Culture=neutral, PublicKeyToken=7e34167dcc6d6d8c'.
[12/2/2019 10:44:26 PM] Stopping JobHost

Do you have any suggestion to solve that problems?

[brettsam]

@soninaren -- Did bundles change?
@alrod -- Any idea on the error from SB?

[soninaren]

@hci2 Can you share the host.json configuration?

Are the following 2 conditions meeting for your function app?

1. Does your host.json have the following configuration.

{
    "version": "2.0",
    "extensionBundle": {
        "id": "Microsoft.Azure.Functions.ExtensionBundle",
        "version": "[1.*, 2.0.0)"
    }
}

2. Do have a bin folder at the root with binaries inside of it?

If you the above 2 conditions are met, then deleting the bin folder should resolve this issue.

[hci2]

@soninaren

1. Yes it does. For more details look below.

Here is the structure of my hostfile:

{
  "version": "2.0",
  "functionTimeout": "00:10:00",
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[1.*, 2.0.0)"
  },
  "serviceBus": {
    "maxConcurrentCalls": 100,
    "prefetchCount": 50,
    "autoRenewTimeout": "00:01:00",
    "autoComplete": true
  }
}

I do have a bin folder in D:\home\site\wwwroot.

I tried to delete the bin directory and restart the Azure Functions via Azure Portal but I am not allowed to do that, see screenshot below:

If I try to delete with FTP (FileZilla) I do not see the same structure as with kudu console.

I will keep trying to delete it somehow and keep you updated, if I know more about it.

Do you have any suggestions how to remove it? Should I create a completely new Azure Function?

I thank you for your effor and wish you a good afternoon!

[soninaren]

If function app is configured to "Run from package" then you won't be able to delete the bin folder. When you use run from package, a zip file is mounted into the wwwroot path. In that case you would have to do a deployment where the generated zip file does not comtain the bin folder.

https://docs.microsoft.com/en-us/azure/azure-functions/run-functions-from-deployment-package

[hci2]

Thank you the the hint. It did not know that before.

I have set the Application Settings "WEBSITE_RUN_FROM_PACKAGE" to 1 and therefore I deployed a new version without bin directory and now it is working again. Thank you really much.

Detailed procedure for Azure Functions (Windows, Code, Java):

1. mvn clean install or mvn clean package
2. delete bin directory from <application_directory>/target/azure_function/<function_name>/
3. mvn azure-functions:deploy

PS: I am using maven plugin for Azure Functions.

pom.xml

<plugin>
        <groupId>com.microsoft.azure</groupId>
        <artifactId>azure-functions-maven-plugin</artifactId>
<plugin>

My concern is that I have to do that behavior every deployment from now on. The workaround is great but I think that this faulty bin directory issue should be fixed in the soon future!

Anyways I really thank you for your help!

[hci2]

SUPPLEMENT: The same steps (delete bin directory) are also necessary when you want to run the Azure Function locally.

Hopefully this strange bug with the creation of a faulty bin directory will be fixed in the near future!

[soninaren]

cc: @amamounelsayed

[amamounelsayed]

@hci2 Thank you for your help. microsoft/azure-maven-plugins#929

[Flanker32]

@hci2 Thanks for your feedback and sorry for the inconvenience, besides, could you please try mvn clean package the project with function maven plugin 1.3.4? As we add the check for function bundles in 1.3.4 and should not generate bin folders, thanks again for your help~

[hci2]

@Flanker32:

Thank you for your hint but I am already using version 1.3.4 and it is still generating the bin folders.

Excerpt of my pom.xml (some appSettings are removed to stay anonymous):

<properties>
 <azure.functions.maven.plugin.version>1.3.4</azure.functions.maven.plugin.version>
 <azure.functions.java.library.version>1.3.0</azure.functions.java.library.version>
</properties>
...
<dependency>
            <groupId>com.microsoft.azure.functions</groupId>
            <artifactId>azure-functions-java-library</artifactId>
            <version>${azure.functions.java.library.version}</version>
</dependency>
...
<plugins>
            <plugin>
                <groupId>com.microsoft.azure</groupId>
                <artifactId>azure-functions-maven-plugin</artifactId>
                <version>${azure.functions.maven.plugin.version}</version>
                <configuration>
                    <resourceGroup>${functionResourceGroup}</resourceGroup>
                    <appName>${functionAppName}</appName>
                    <region>${functionAppRegion}</region>
                    <authentication>
                        <serverId>azure_auth_danglp_master</serverId>
                    </authentication>
                    <appSettings>
                        <!-- Run Azure Function from package file by default -->
                        <property>
                            <name>WEBSITE_RUN_FROM_PACKAGE</name>
                            <value>1</value>
                        </property>
                        <property>
                            <name>FUNCTIONS_EXTENSION_VERSION</name>
                            <value>~2</value>
                        </property>
                        <property>
                            <name>WEBSITE_HTTPLOGGING_RETENTION_DAYS</name>
                            <value>30</value>
                        </property>
                        <property>
                            <name>WEBSITE_TIME_ZONE</name>
                            <value>UTC</value>
                        </property>
                    </appSettings>
                </configuration>
                <executions>
                    <execution>
                        <id>package-functions</id>
                        <goals>
                            <goal>package</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-resources-plugin</artifactId>
                <version>3.1.0</version>
                <executions>
                    <execution>
                        <id>copy-resources</id>
                        <phase>package</phase>
                        <goals>
                            <goal>copy-resources</goal>
                        </goals>
                        <configuration>
                            <overwrite>true</overwrite>
                            <outputDirectory>${stagingDirectory}</outputDirectory>
                            <resources>
                                <resource>
                                    <directory>${project.basedir}/src/main/resources/azure</directory>
                                    <includes>
                                        <include>host.json</include>
                                        <include>local.settings.json</include>
                                    </includes>
                                </resource>
                            </resources>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
                <version>3.1.1</version>
                <executions>
                    <execution>
                        <id>copy-dependencies</id>
                        <phase>prepare-package</phase>
                        <goals>
                            <goal>copy-dependencies</goal>
                        </goals>
                        <configuration>
                            <outputDirectory>${stagingDirectory}/lib</outputDirectory>
                            <overWriteReleases>false</overWriteReleases>
                            <overWriteSnapshots>false</overWriteSnapshots>
                            <overWriteIfNewer>true</overWriteIfNewer>
                            <includeScope>runtime</includeScope>
                            <excludeArtifactIds>azure-functions-java-library</excludeArtifactIds>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <artifactId>maven-clean-plugin</artifactId>
                <version>3.1.0</version>
                <configuration>
                    <filesets>
                        <fileset>
                            <directory>obj</directory>
                        </fileset>
                    </filesets>
                </configuration>
            </plugin>
        </plugins>