Time out when i access Azure Data Lake Gen2 in Azure Function with ADLS java sdk V12

[joshfree]

Ported from Azure/azure-functions-java-library#113 after offline discussion with CSE

---

gjjtip commented on May 4

My function code as below:

package org.example;

import com.azure.core.http.rest.PagedIterable;
import com.azure.storage.common.StorageSharedKeyCredential;
import com.azure.storage.file.datalake.DataLakeFileSystemClient;
import com.azure.storage.file.datalake.DataLakeServiceClient;
import com.azure.storage.file.datalake.DataLakeServiceClientBuilder;
import com.azure.storage.file.datalake.models.ListPathsOptions;
import com.azure.storage.file.datalake.models.PathItem;
import com.microsoft.azure.functions.*;
import com.microsoft.azure.functions.annotation.AuthorizationLevel;
import com.microsoft.azure.functions.annotation.FunctionName;
import com.microsoft.azure.functions.annotation.HttpTrigger;

import java.util.Iterator;
import java.util.Optional;

/**
 * Azure Functions with HTTP Trigger.
 */
public class Function {

    public static String accountName = "***";
    public static String accountKey = "***";

    @FunctionName("HttpExample")
    public HttpResponseMessage run(
            @HttpTrigger(name = "req", methods = {HttpMethod.GET, HttpMethod.POST}, authLevel = AuthorizationLevel.ANONYMOUS) HttpRequestMessage<Optional<String>> request,
            final ExecutionContext context) {
        context.getLogger().info("Java HTTP trigger processed a request.+++++");

        DataLakeServiceClient dataLakeServiceClient = GetDataLakeServiceClient(accountName, accountKey);
        DataLakeFileSystemClient dataLakeFileSystemClient = GetFileSystem(dataLakeServiceClient);

        String name = ListFilesInDirectory(dataLakeFileSystemClient, context);

        if (name == null) {
            return request.createResponseBuilder(HttpStatus.BAD_REQUEST).body("Please pass a name on the query string or in the request body").build();
        } else {
            return request.createResponseBuilder(HttpStatus.OK).body("List File Names:, " + name).build();
        }
    }

    static public DataLakeServiceClient GetDataLakeServiceClient
            (String accountName, String accountKey) {

        StorageSharedKeyCredential sharedKeyCredential =
                new StorageSharedKeyCredential(accountName, accountKey);

        DataLakeServiceClientBuilder builder = new DataLakeServiceClientBuilder();

        builder.credential(sharedKeyCredential);
        builder.endpoint("https://" + accountName + ".dfs.core.windows.net");

        return builder.buildClient();
    }

    static public DataLakeFileSystemClient GetFileSystem
            (DataLakeServiceClient serviceClient) {

        return serviceClient.getFileSystemClient("test");
    }

    static public String ListFilesInDirectory(DataLakeFileSystemClient fileSystemClient, ExecutionContext context) {

        ListPathsOptions options = new ListPathsOptions();
        options.setPath("");
        fileSystemClient.listPaths().forEach( path -> context.getLogger().info(path.getName()));

        return "ABC";
    }
}

The java sdk link: https://azuresdkdocs.blob.core.windows.net/$web/java/azure-storage-file-datalake/12.0.0-preview.6/index.html

Based on my own troubleshooting, the code always hangs on the line :
fileSystemClient.listPaths().forEach( path -> context.getLogger().info(path.getName()));
No any responses here. I tested above code outside Azure function, let's say the Main method,everything works. So i think the code is fine. Anyone met the same issue as same as me. Is there any restrictions in the Azure Function to access ADLS gen2 with java sdk? Please help.

---

DracoTaffi commented on Jun 17

I have the same issue ..
I can get the FileSystemClient and DirectoryClient objects but time out checking the existence whatever the value of this timeout:

DataLakeFileSystemClient adlsfsClient = adlsClient.getFileSystemClient(fileSystem);
DataLakeDirectoryClient adlsRootDirClient = adlsfsClient.getDirectoryClient("myrootdir");
Duration timeout = Duration.ofMillis(10000);
if (!adlsRootDirClient.existsWithResponse(timeout, Context.NONE).getValue()) {
    adlsRootDirClient.create();
}

Same as you, running this code from my local PC works fine !!

[joshfree]

/cc @amishra-dev @gapra-msft @rickle-msft

[rickle-msft]

@joshfree There are some fixes that need to go into the Azure Functions workers to unblock some of this stuff, right? Do we know the status of that?

[shaily-b]

I am facing the same issue with DataLakeFileSystemClient and BlobContainerClient also when working with blob storage. It is getting stuck at listing files when accessed through Azure Function (Event Grid Trigger) and then failing with Timeout exception, while it works perfectly fine in my local.

blobContainerClient.listBlobs()
.forEach(pathItem -> System.out.println("Blob name: " + pathItem.getName()));

Do we have any update on this issue?

[rickle-msft]

@ShailyBhagwani You should be able to track the related fix for Azure Functions here

[scgbear]

Internal Tracking

devdivcsef 371565

[scgbear]

@rickle-msft the dependent azure functions issue has been closed. This mean this issue has been resolved?

[kasobol-msft]

I tried the sample provided by customer and deployed it into Functions. It seems to be working, i.e. no timeout observed. Closing.

I used

        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-storage-file-datalake</artifactId>
            <version>12.2.0</version>
        </dependency>