Minio 6.0.2: Unexpected Behavior with Unavailable Endpoints

[teun2408]

Issue Summary:

Upgrading from Minio .NET SDK version 6.0.1 to 6.0.2 has introduced unexpected behavior when interacting with non-existent Minio server endpoints. Specifically, methods like BucketExistsAsync and GetObjectAsync do not behave as expected when provided with endpoints that are not running Minio servers. Instead of throwing exceptions or indicating errors, they return unexpected results, such as always returning true for bucket existence checks and returning empty streams for object downloads.

Steps to Reproduce:

1. Use Minio .NET SDK version 6.0.2.
2. Configure the SDK with an endpoint that does not host a Minio server (e.g., "127.0.0.1:11111").
3. Use the BucketExistsAsync method to check for the existence of a bucket that doesn't exist.
4. Use the GetObjectAsync method to download an object from a non-existent bucket.

using Minio;

var s3Client = new MinioClient()
    .WithEndpoint("127.0.0.1:11111")
    .WithCredentials("accesskey", "secretkey")
    .Build();

//Should throw an exception, but it does not in 6.0.2, in 6.0.1 it throws a System.AggregateException
//Instead, in 6.0.2 it just returns true even though the Minio server does not exist, nor the bucket.
bool bucketExists = s3Client.BucketExistsAsync(new Minio.DataModel.Args.BucketExistsArgs().WithBucket("bucketname")).Result;

if (bucketExists)
{
    Console.WriteLine("Bucket exists according to Minio");
}

MemoryStream stream = new MemoryStream();

// An exception should be thrown here, but it is not, instead, it returns an empty stream even though neither the server, bucket, or object exists.
await s3Client.GetObjectAsync(new Minio.DataModel.Args.GetObjectArgs().WithBucket("bucketname").WithObject("objectname").WithCallbackStream(x => x.CopyTo(stream)));

Console.WriteLine($"Stream length: {stream.Length}");

Expected Behavior:

• The BucketExistsAsync method should throw an exception or return false when provided with an endpoint that does not host a Minio server.
• The GetObjectAsync method should throw an exception when attempting to download an object from a non-existent bucket.

Actual Behavior:

• When using BucketExistsAsync with a non-existent Minio server endpoint, it consistently returns true, incorrectly indicating the existence of the bucket.
• When using GetObjectAsync with a non-existent bucket or object, it returns an empty stream instead of throwing an exception or indicating an error.

Environment Details:

• Minio .NET SDK version: 6.0.2
• Operating System: Windows 11
• .NET 8

[cvetomir-todorov]

Happens to me as well. Works with 6.0.1, breaks with 6.0.2. I am using Ubuntu 22.04 and .NET 8.

[OK111]

Also PutObjectAsync doesn't throw any errors in 6.0.2 .NET 8

[guillaumeagile]

The same. I tried to check the behavior of the driver with test containers, and got the same issue. I finally realized that the problem doesn't come from TestContainers, but from the driver.

[guillaumeagile]

Observed exactly that weird behavior when the endpoint is unreachable
like if the driver was talking to a "ghost" server that says yes to everything and that acts like a black hole when you upload to it.

Using .Net 8, TestContainers , xUnit and 6.0.2
service.AddMinio(configureClient => configureClient .WithEndpoint(uri) .WithCredentials(MinIoAdapter.AccessKey, MinIoAdapter.SecretKey) .WithSSL(false) .Build() );
where uri is not pointing to a running Minio instance.

await _minioClient.BucketExistsAsync(beArgs).ConfigureAwait(false);
or await _minioClient.PresignedGetObjectAsync(presignedGetObjectArgs).ConfigureAwait(false)
always returns something (with an empty container)

_minioClient.GetObjectAsync(getObjectArgs).ConfigureAwait(true)
always returns a 0 byte length memoryStream or file, even if I has uploaded an object before (with the right name)

I'm initializing my container with

_minioContainer = new MinioBuilder() .WithUsername(MinIoAdapter.AccessKey) .WithPassword(MinIoAdapter.SecretKey) .WithPortBinding(9000, true) .WithCleanUp(true) .Build();
it's also obvious when I remove the .WithUsername(MinIoAdapter.AccessKey) .WithPassword

[holterbades]

Imho this is a critical problem and this Issue should get the name "No error handling" because there is no actual error handling active anymore.

I encountered the following issues:

When i download a file with valid bucketName and objectName but the user/access-token does not have permission to download the object (e.g. due to a policy) or a wrong region was set in the client, no exception is thrown. So i can't verify that the download failed. The funny thing is, that instead of the object-content on the storage, the xml error response gets written to the output file.

var getObject = new GetObjectArgs()
    .WithBucket(bucketName)
    .WithObject(objectName)
    .WithFile("C:\\temp\\test2.txt");
var stat = await minio.GetObjectAsync(getObject);

Similar to this example, if i stream the object content directly to another destination (e.g. another S3 object store), i also get the error message content instead of an exception streamed to the destination. Furthermore, if i encounter an exception in the streaming action like outlined below, this exception is also catched and not delegated to the user code, which also makes it impossible to handle errors.

var getObject = new GetObjectArgs()
    .WithBucket(bucketName)
    .WithObject(objectName)
    .WithCallbackStream(stream =>
    {
        throw new Exception("Error uploading file to storage.");
    });
var stat = await minio.GetObjectAsync(getObject);

Imagine you use this action to stream the blob directly to an deep archive with retention for 7 years - in this case you get files with error messages instead of your intended object content - which you can't delete because of retention policies set on the target.

The only way you could verify that an error occoured is by validating the ObjectStat.ExtraHeaders returned by GetObjectAsync(...). But this clearly can't be a solution because it is provider dependent and would be a breaking change for clients out there.

I traced back the problem to changes in the commit 9772a373b92d81808118edbdbd653d301f4502ac, but I don't know what the intention for those changes was because it's not well documented neither in the commit nor in the linked issue (#987). But the problem is, that in this commit, the method call to HandleIfErrorResponse was unintentionally removed (this method has no other calls and if this removal was intended, the method HandleIfErrorResponse should have also been removed).

I opened a PullRequest #1057 to fix this issue.

[holterbades]

I think, the other problems mentioned in this issue should also get resolved with my fix, because this method ExecuteTaskAsync(...) -> ExecuteTaskCoreAsync(...) is the core method used by all operations.