Skip to content

Commit 30d498f

Browse files
mocsharpdependabot[bot]lillie-dae
authored
Release/0.2.6 (#76)
* Storage service extension for health check service (#46) * Add health check for MinIO * Update packages & licenses Signed-off-by: Victor Chang <vicchang@nvidia.com> * Bump codecov/codecov-action from 2 to 3 (#37) Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 2 to 3. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md) - [Commits](codecov/codecov-action@v2...v3) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-dotnet from 1 to 2 (#36) Bumps [actions/setup-dotnet](https://github.com/actions/setup-dotnet) from 1 to 2. - [Release notes](https://github.com/actions/setup-dotnet/releases) - [Commits](actions/setup-dotnet@v1...v2) --- updated-dependencies: - dependency-name: actions/setup-dotnet dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump jungwinter/split from 1 to 2 (#31) Bumps [jungwinter/split](https://github.com/jungwinter/split) from 1 to 2. - [Release notes](https://github.com/jungwinter/split/releases) - [Commits](winterjung/split@v1...v2) --- updated-dependencies: - dependency-name: jungwinter/split dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump gittools/actions from 0.9.11 to 0.9.13 (#33) Bumps [gittools/actions](https://github.com/gittools/actions) from 0.9.11 to 0.9.13. - [Release notes](https://github.com/gittools/actions/releases) - [Commits](GitTools/actions@v0.9.11...v0.9.13) --- updated-dependencies: - dependency-name: gittools/actions dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-java from 1 to 3 (#34) Bumps [actions/setup-java](https://github.com/actions/setup-java) from 1 to 3. - [Release notes](https://github.com/actions/setup-java/releases) - [Commits](actions/setup-java@v1...v3) --- updated-dependencies: - dependency-name: actions/setup-java dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 2 to 3 (#51) Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v2...v3) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/download-artifact from 2 to 3 (#48) Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 2 to 3. - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@v2...v3) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/upload-artifact from 2.3.1 to 3.1.0 (#49) Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2.3.1 to 3.1.0. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@v2.3.1...v3.1.0) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/cache from 2.1.7 to 3.0.8 (#50) Bumps [actions/cache](https://github.com/actions/cache) from 2.1.7 to 3.0.8. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@v2.1.7...v3.0.8) --- updated-dependencies: - dependency-name: actions/cache dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update ci.yml * Add minio admin heath check (#55) * add minio admin health check Signed-off-by: Lillie Dae <lillie.dae@answerdigital.com> * Fix typo * Allow enable/disable service/admin health checks Signed-off-by: Victor Chang <vicchang@nvidia.com> * Minio admin healthcheck fix (#61) * minor fixes to admin health checks Signed-off-by: Lillie Dae <lillie.dae@answerdigital.com> * Log temporary credential policy for debugging Signed-off-by: Victor Chang <vicchang@nvidia.com> * Update policy to return distinct paths Signed-off-by: Victor Chang <vicchang@nvidia.com> * Fix unit test Signed-off-by: Victor Chang <vicchang@nvidia.com> * Enable concurrent calls to create MinIo clients Signed-off-by: Victor Chang <vicchang@nvidia.com> * Update packages & licenses Signed-off-by: Victor Chang <vicchang@nvidia.com> * Update packages & licenses Signed-off-by: Victor Chang <vicchang@nvidia.com> * Update packages & licenses Signed-off-by: Victor Chang <vicchang@nvidia.com> Signed-off-by: Victor Chang <vicchang@nvidia.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Lillie Dae <lillie.dae@answerdigital.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Lillie Dae <61380713+lillie-dae@users.noreply.github.com>
1 parent 8663a10 commit 30d498f

14 files changed

+87
-125
lines changed

‎doc/dependency_decisions.yml‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,21 +4,21 @@
44
- :who: mocsharp
55
:why: Apache-2.0 (https://github.com/aws/aws-sdk-net/raw/master/License.txt)
66
:versions:
7-
- 3.7.12.26
7+
- 3.7.13.5
88
:when: 2022-08-29 18:11:12.923214877 Z
99
- - :approve
1010
- AWSSDK.S3
1111
- :who: mocsharp
1212
:why: Apache-2.0 (https://github.com/aws/aws-sdk-net/raw/master/License.txt)
1313
:versions:
14-
- 3.7.9.44
14+
- 3.7.9.54
1515
:when: 2022-08-29 18:11:13.354973002 Z
1616
- - :approve
1717
- AWSSDK.SecurityToken
1818
- :who: mocsharp
1919
:why: Apache-2.0 (https://github.com/aws/aws-sdk-net/raw/master/License.txt)
2020
:versions:
21-
- 3.7.1.190
21+
- 3.7.1.200
2222
:when: 2022-08-16 18:11:13.781079769 Z
2323
- - :approve
2424
- Ardalis.GuardClauses
@@ -144,14 +144,14 @@
144144
- :who: mocsharp
145145
:why: MIT (https://github.com/dotnet/aspnetcore/raw/main/LICENSE.txt)
146146
:versions:
147-
- 6.0.8
147+
- 6.0.9
148148
:when: 2022-08-29 18:11:22.090772006 Z
149149
- - :approve
150150
- Microsoft.Extensions.Diagnostics.HealthChecks.Abstractions
151151
- :who: mocsharp
152152
:why: MIT (https://github.com/dotnet/aspnetcore/raw/main/LICENSE.txt)
153153
:versions:
154-
- 6.0.8
154+
- 6.0.9
155155
:when: 2022-08-29 18:11:22.090772006 Z
156156
- - :approve
157157
- Microsoft.Extensions.FileProviders.Abstractions
@@ -200,7 +200,7 @@
200200
- :who: mocsharp
201201
:why: MIT (https://github.com/dotnet/runtime/raw/main/LICENSE.TXT)
202202
:versions:
203-
- 6.0.1
203+
- 6.0.2
204204
:when: 2022-08-29 18:11:25.167886026 Z
205205
- - :approve
206206
- Microsoft.Extensions.Logging.Configuration
@@ -487,14 +487,14 @@
487487
- :who: mocsharp
488488
:why: MIT ( https://github.com/TestableIO/System.IO.Abstractions/raw/main/LICENSE)
489489
:versions:
490-
- 17.1.1
490+
- 17.2.3
491491
:when: 2022-08-16 18:11:44.550586685 Z
492492
- - :approve
493493
- System.IO.Abstractions.TestingHelpers
494494
- :who: mocsharp
495495
:why: MIT ( https://github.com/TestableIO/System.IO.Abstractions/raw/main/LICENSE)
496496
:versions:
497-
- 17.1.1
497+
- 17.2.3
498498
:when: 2022-08-16 18:11:44.984891276 Z
499499
- - :approve
500500
- System.IO.Compression

‎src/Plugins/AWSS3/Monai.Deploy.Storage.AWSS3.csproj‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,8 +49,8 @@
4949

5050
<ItemGroup>
5151
<PackageReference Include="Ardalis.GuardClauses" Version="4.0.1" />
52-
<PackageReference Include="AWSSDK.S3" Version="3.7.9.44" />
53-
<PackageReference Include="AWSSDK.SecurityToken" Version="3.7.1.190" />
52+
<PackageReference Include="AWSSDK.S3" Version="3.7.9.54" />
53+
<PackageReference Include="AWSSDK.SecurityToken" Version="3.7.1.200" />
5454
<PackageReference Include="Microsoft.Extensions.Logging" Version="6.0.0" />
5555
<PackageReference Include="Microsoft.Extensions.Options" Version="6.0.0" />
5656
<PackageReference Include="Newtonsoft.Json" Version="13.0.1" />

‎src/Plugins/MinIO/ConfigurationKeys.cs‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,7 @@ namespace Monai.Deploy.Storage.MinIO
1818
{
1919
internal static class ConfigurationKeys
2020
{
21-
public static readonly string StorageServiceName = "MinIo";
22-
public static readonly string StorageAdminServiceName = "MinIo Admin";
21+
public static readonly string StorageServiceName = "minio";
2322

2423
public static readonly string EndPoint = "endpoint";
2524
public static readonly string AccessKey = "accessKey";

‎src/Plugins/MinIO/HealthCheckBuilder.cs‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -25,25 +25,25 @@ public class HealthCheckBuilder : HealthCheckRegistrationBase
2525
{
2626
public override IHealthChecksBuilder ConfigureAdminHealthCheck(IHealthChecksBuilder builder, HealthStatus? failureStatus = null, IEnumerable<string>? tags = null, TimeSpan? timeout = null) =>
2727
builder.Add(new HealthCheckRegistration(
28-
$"{ConfigurationKeys.StorageAdminServiceName}",
28+
ConfigurationKeys.StorageServiceName,
2929
serviceProvider =>
3030
{
31-
var logger = serviceProvider.GetRequiredService<ILogger<MinIoAdminHealthCheck>>();
32-
var storageAdminService = serviceProvider.GetRequiredService<IStorageAdminService>();
33-
return new MinIoAdminHealthCheck(storageAdminService, logger);
31+
var logger = serviceProvider.GetRequiredService<ILogger<MinIoHealthCheck>>();
32+
var minioClientFactory = serviceProvider.GetRequiredService<IMinIoClientFactory>();
33+
return new MinIoHealthCheck(minioClientFactory, logger);
3434
},
3535
failureStatus,
3636
tags,
3737
timeout));
3838

3939
public override IHealthChecksBuilder ConfigureHealthCheck(IHealthChecksBuilder builder, HealthStatus? failureStatus = null, IEnumerable<string>? tags = null, TimeSpan? timeout = null) =>
4040
builder.Add(new HealthCheckRegistration(
41-
ConfigurationKeys.StorageServiceName,
41+
$"{ConfigurationKeys.StorageServiceName}-admin",
4242
serviceProvider =>
4343
{
44-
var logger = serviceProvider.GetRequiredService<ILogger<MinIoHealthCheck>>();
45-
var minioClientFactory = serviceProvider.GetRequiredService<IMinIoClientFactory>();
46-
return new MinIoHealthCheck(minioClientFactory, logger);
44+
var logger = serviceProvider.GetRequiredService<ILogger<MinIoAdminHealthCheck>>();
45+
var storageAdminService = serviceProvider.GetRequiredService<IStorageAdminService>();
46+
return new MinIoAdminHealthCheck(storageAdminService, logger);
4747
},
4848
failureStatus,
4949
tags,

‎src/Plugins/MinIO/LoggerMethods.cs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,5 +31,8 @@ public static partial class LoggerMethods
3131

3232
[LoggerMessage(EventId = 20003, Level = LogLevel.Error, Message = "Health check failure.")]
3333
public static partial void HealthCheckError(this ILogger logger, Exception ex);
34+
35+
[LoggerMessage(EventId = 20004, Level = LogLevel.Debug, Message = "Temporary credential policy={policy}.")]
36+
public static partial void TemporaryCredentialPolicy(this ILogger logger, string policy);
3437
}
3538
}

‎src/Plugins/MinIO/MinIoClientFactory.cs‎

Lines changed: 21 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
* limitations under the License.
1515
*/
1616

17+
using System.Collections.Concurrent;
1718
using Amazon.SecurityToken.Model;
1819
using Ardalis.GuardClauses;
1920
using Microsoft.Extensions.Options;
@@ -25,7 +26,7 @@ namespace Monai.Deploy.Storage.MinIO
2526
public class MinIoClientFactory : IMinIoClientFactory
2627
{
2728
private static readonly string DefaultClient = "_DEFAULT_";
28-
private readonly Dictionary<string, MinioClient> _clients;
29+
private readonly ConcurrentDictionary<string, MinioClient> _clients;
2930

3031
private StorageServiceConfiguration Options { get; }
3132

@@ -38,22 +39,19 @@ public MinIoClientFactory(IOptions<StorageServiceConfiguration> options)
3839

3940
Options = configuration;
4041

41-
_clients = new Dictionary<string, MinioClient>();
42+
_clients = new ConcurrentDictionary<string, MinioClient>();
4243
}
4344

4445
public MinioClient GetClient()
4546
{
46-
if (_clients.ContainsKey(DefaultClient))
47-
{
48-
return _clients[DefaultClient];
49-
}
50-
51-
var accessKey = Options.Settings[ConfigurationKeys.AccessKey];
52-
var accessToken = Options.Settings[ConfigurationKeys.AccessToken];
53-
var client = CreateClient(accessKey, accessToken);
54-
55-
_clients[DefaultClient] = client.Build();
56-
return _clients[DefaultClient];
47+
return _clients.GetOrAdd(DefaultClient, _ =>
48+
{
49+
var accessKey = Options.Settings[ConfigurationKeys.AccessKey];
50+
var accessToken = Options.Settings[ConfigurationKeys.AccessToken];
51+
var client = CreateClient(accessKey, accessToken);
52+
53+
return client.Build();
54+
});
5755
}
5856

5957
public MinioClient GetClient(Credentials credentials)
@@ -68,21 +66,20 @@ public MinioClient GetClient(Credentials credentials, string region)
6866
Guard.Against.NullOrWhiteSpace(credentials.SecretAccessKey, nameof(credentials.SecretAccessKey));
6967
Guard.Against.NullOrWhiteSpace(credentials.SessionToken, nameof(credentials.SessionToken));
7068

71-
if (_clients.ContainsKey(credentials.SessionToken))
69+
return _clients.GetOrAdd(credentials.SessionToken, _ =>
7270
{
73-
return _clients[credentials.SessionToken];
74-
}
71+
var client = CreateClient(credentials.AccessKeyId, credentials.SecretAccessKey);
72+
client.WithSessionToken(credentials.SessionToken);
7573

76-
var client = CreateClient(credentials.AccessKeyId, credentials.SecretAccessKey);
77-
client.WithSessionToken(credentials.SessionToken);
74+
if (!string.IsNullOrWhiteSpace(region))
75+
{
76+
client.WithRegion(region);
77+
}
78+
79+
return client.Build();
80+
});
7881

79-
if (!string.IsNullOrWhiteSpace(region))
80-
{
81-
client.WithRegion(region);
82-
}
8382

84-
_clients[DefaultClient] = client.Build();
85-
return _clients[DefaultClient];
8683
}
8784

8885
private MinioClient CreateClient(string accessKey, string accessToken)

‎src/Plugins/MinIO/MinIoStorageService.cs‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -203,6 +203,7 @@ public async Task<Credentials> CreateTemporaryCredentialsAsync(string bucketName
203203

204204
var policyString = JsonConvert.SerializeObject(policy, Formatting.None, new JsonSerializerSettings { NullValueHandling = NullValueHandling.Ignore });
205205

206+
_logger.TemporaryCredentialPolicy(policyString);
206207
var assumeRoleRequest = new AssumeRoleRequest
207208
{
208209
DurationSeconds = durationSeconds,

‎src/Plugins/MinIO/Monai.Deploy.Storage.MinIO.csproj‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@
4848

4949
<ItemGroup>
5050
<PackageReference Include="Ardalis.GuardClauses" Version="4.0.1" />
51-
<PackageReference Include="AWSSDK.SecurityToken" Version="3.7.1.190" />
51+
<PackageReference Include="AWSSDK.SecurityToken" Version="3.7.1.200" />
5252
<PackageReference Include="Microsoft.Extensions.Logging" Version="6.0.0" />
5353
<PackageReference Include="Microsoft.Extensions.Options" Version="6.0.0" />
5454
<PackageReference Include="Minio" Version="4.0.5" />

‎src/Plugins/MinIO/Tests/Monai.Deploy.Storage.MinIO.Tests.csproj‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@
2828
<PackageReference Include="Microsoft.Extensions.Hosting" Version="6.0.1" />
2929
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.1" />
3030
<PackageReference Include="Moq" Version="4.18.2" />
31-
<PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="17.1.1" />
31+
<PackageReference Include="System.IO.Abstractions.TestingHelpers" Version="17.2.3" />
3232
<PackageReference Include="xunit" Version="2.4.2" />
3333
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.5">
3434
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>

‎src/S3Policy/PolicyExtensions.cs‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ public static Policy ToPolicy(string? bucketName, string? folderName)
5353
{
5454
StringEquals = new StringEquals
5555
{
56-
S3Prefix = pathList.ToArray(),
56+
S3Prefix = pathList.Distinct().ToArray(),
5757
S3Delimiter = new string[] { "/" }
5858
}
5959
}
@@ -68,7 +68,7 @@ public static Policy ToPolicy(string? bucketName, string? folderName)
6868
{
6969
StringEquals = new StringEquals
7070
{
71-
S3Prefix = new string[] {$"{folderName}/*" }
71+
S3Prefix = new string[] {$"{folderName}/*", folderName }
7272
}
7373
}
7474
},
@@ -128,7 +128,7 @@ public static Policy ToPolicy(PolicyRequest[] policyRequests)
128128
{
129129
S3Prefix = policyRequests
130130
.Select(pr => $"{pr.FolderName}/*")
131-
.Union( policyRequests.Select(pr => $"{pr.FolderName}"))
131+
.Union( policyRequests.Select(pr => pr.FolderName))
132132
.Distinct().ToArray()
133133
}
134134
}

0 commit comments

Comments
 (0)