[BUG] Dependency Conflict Between Worker and Microsoft-Graph

[sainath341]

When using batch request from microsoft-graph sdk (for java) I am getting below error because of dependency conflict between function and java worker.
Exception: NoSuchMethodError: 'com.google.gson.JsonElement com.google.gson.JsonParser.parseString(java.lang.String)'

Investigative information

Followed this issue #381 and tried both the java version but got the same error.

Please provide the following:

• Timestamp: 2021-03-18T14:43:05.185
• Function versionDetails: 3.0.15405 Commit hash: c696322564f1f9dc9557bfa495c0485ddf71eeef
• Function name(s) (as appropriate):
• Invocation ID: 4cd2cc7be2a7a2d3aad6c3aba166061cc1bfefdb6a823d52ba5bec492ffb1671
• Region: central-us

Repro steps

Follow this example Simple batching example or use below code and deploy in azure.

package net.comp.function;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;

import com.google.gson.Gson;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
import com.microsoft.azure.functions.ExecutionContext;
import com.microsoft.azure.functions.HttpMethod;
import com.microsoft.azure.functions.HttpRequestMessage;
import com.microsoft.azure.functions.HttpResponseMessage;
import com.microsoft.azure.functions.HttpStatus;
import com.microsoft.azure.functions.annotation.AuthorizationLevel;
import com.microsoft.azure.functions.annotation.FunctionName;
import com.microsoft.azure.functions.annotation.HttpTrigger;
import com.microsoft.graph.auth.confidentialClient.ClientCredentialProvider;
import com.microsoft.graph.auth.enums.NationalCloud;
import com.microsoft.graph.content.MSBatchRequestContent;
import com.microsoft.graph.content.MSBatchRequestStep;
import com.microsoft.graph.content.MSBatchResponseContent;
import com.microsoft.graph.httpcore.HttpClients;
import com.microsoft.graph.httpcore.ICoreAuthenticationProvider;
import com.microsoft.graph.models.extensions.IGraphServiceClient;
import com.microsoft.graph.requests.extensions.GraphServiceClient;
import com.microsoft.graph.serializer.ISerializer;

import okhttp3.MediaType;
import okhttp3.OkHttpClient;
import okhttp3.Request;
import okhttp3.RequestBody;
import okhttp3.Response;

public class Function {
    @FunctionName("employee")
    public HttpResponseMessage run(@HttpTrigger(name = "req", methods = { HttpMethod.GET,
            HttpMethod.POST }, authLevel = AuthorizationLevel.ANONYMOUS) HttpRequestMessage<Optional<String>> request,
            final ExecutionContext context) throws IOException {
        context.getLogger().info("Java HTTP trigger processed a request.");

        String query = request.getQueryParameters().get("empId");
        String id = request.getBody().orElse(query);
        String TENANT_ID = System.getenv("TENANT_ID");
        String CLIENT_ID = System.getenv("CLIENT_ID");
        String CLIENT_SECRET = System.getenv("CLIENT_SECRET");
        List<String> SCOPES = new ArrayList<>(Arrays.asList("https://graph.microsoft.com/.default"));
        ClientCredentialProvider authProvider = new ClientCredentialProvider(CLIENT_ID, SCOPES, CLIENT_SECRET,
                TENANT_ID, NationalCloud.Global);

        IGraphServiceClient graphClient = GraphServiceClient.builder().authenticationProvider(authProvider)
                .buildClient();
        try {
            return request.createResponseBuilder(HttpStatus.OK).body(getUserIds(id.split(","), graphClient)).build();
        } catch (Exception e) {
            return request.createResponseBuilder(HttpStatus.INTERNAL_SERVER_ERROR).body("failed").build();
        }
    }

    public String getUserIds(String[] empList, IGraphServiceClient graphClient) throws IOException {
        Map<String, String> empMap = new HashMap<>();
        MSBatchRequestContent batchRequestContent = new MSBatchRequestContent();
        for (String emp : empList) {
            String filter = "employeeId eq '" + emp + "'";
            MSBatchRequestStep batchRequestStep = new MSBatchRequestStep(emp,
                    graphClient.users().buildRequest().filter(filter).select("id").top(1)
                            .withHttpMethod(com.microsoft.graph.http.HttpMethod.GET).getHttpRequest(),
                    null);
            batchRequestContent.addBatchRequestStep(batchRequestStep);
        }
        ICoreAuthenticationProvider auth = (ICoreAuthenticationProvider) graphClient.getAuthenticationProvider();
        OkHttpClient client = HttpClients.createDefault(auth);

        String batchContent = batchRequestContent.getBatchRequestContent();
        Request batchRequest = new Request.Builder().url("https://graph.microsoft.com/beta/$batch")
                .post(RequestBody.create(MediaType.parse("application/json"), batchContent)).build();

        Response batchResponse = client.newCall(batchRequest).execute();
        final ISerializer graphSerializer = graphClient.getSerializer();
        MSBatchResponseContent batchResponseContent = new MSBatchResponseContent(batchResponse);
        for (String emp : empList) {
           Response tempRes = batchResponseContent.getResponseById(emp);
            try {
                if (tempRes.isSuccessful()) {
                    JsonObject res = JsonParser.parseString(tempRes.body().string()).getAsJsonObject();
                    String m = res.get("value").getAsJsonArray().size() > 0
                            ? res.get("value").getAsJsonArray().get(0).getAsJsonObject().get("id").getAsString()
                            : "";
                    empMap.put(emp, m);
                } else {
                    empMap.put(emp, null);
                }

            } catch (Exception e) {
                empMap.put(emp, null);
                e.printStackTrace();
            }
        }
        return new Gson().toJson(empMap);
    }
}

Dependencies to include

        <dependency>
            <groupId>com.microsoft.azure.functions</groupId>
            <artifactId>azure-functions-java-library</artifactId>
            <version>${azure.functions.java.library.version}</version>
        </dependency>
        <dependency>
            <groupId>com.microsoft.graph</groupId>
            <artifactId>microsoft-graph</artifactId>
            <version>2.7.1</version>
        </dependency>
        <dependency>
            <groupId>com.microsoft.graph</groupId>
            <artifactId>microsoft-graph-auth</artifactId>
            <version>0.3.0</version>
        </dependency>

I fixed local env by modifying gson dependency in azure-functions-java-worker.jar

Provide the steps required to reproduce the problem:
Just run the function

Expected behavior

Running the application successfully.

Actual behavior

Application crashed with NoSuchMethodError,

Known workarounds

Provide a description of any known workarounds.

• I can change the worker dependencies locally but not in the cloud.

Related information

Provide any related information

• Programming language used : Java
• Links to source : none
• Bindings used : none

[amamounelsayed]

Thank you so much for your message. Can you please share what did you do for "I fixed local env by modifying gson dependency in azure-functions-java-worker.jar"?

Thank you so much

[sainath341]

I opened azure-functions-java-worker.jar and replaced gson class files with gson (2.8.6).

[amamounelsayed]

Thank you so much,
Unfortunately as you mentioned the worker depends on the Gson jar. It is the only one exposed as dependency.
We shaded before but we faced this issue #400

Which sku are you planning to use in the cloud?

[sainath341]

Is sku a service plan type ?
I'm using Standard s1

[amamounelsayed]

Sorry for my delay, Unfortunately to upgrade GSON will need new release there is no ETA for the upgrade, but I was looking for options. As your on Standard S1, you can follow https://github.com/Azure/azure-functions-host/wiki/Deploying-the-Functions-runtime-as-a-private-site-extension

By this way you can have your own jar upgraded. But this will require you for each release to update the host code as you will not be getting it automatically.

Please let us if you have any questions

[sainath341]

Okay thank you @amamounelsayed, As a workaround there is a way to target custom worker in azure which solves my problem temporarily.

Steps :

• copy the workers folder from local (\azure-functions-core-tools\bin\workers) which contains java worker(the modified azure-functions-java-worker.jar, lib and worker.config.json )
• Paste the folder in azure function using kudu ( for ex: D:\home\site>)
• In azure function set the following property in application settings.
  "languageWorkers:workersDirectory": "D:/home/site/workers"

[shreyas-gopalakrishna]

@sainath341 Thank you for the workaround. I will be closing this issue as we are tracking customizing the gson version in the below issue
#424