Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tenant removal (Lombiq Technologies: OCORE-96) #11890

Merged
merged 137 commits into from
Jan 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
137 commits
Select commit Hold shift + click to select a range
8e67b75
Start on tenant removal feature.
jtkech Jun 21, 2022
0d4b03f
Remove extra and bad line of code
jtkech Jun 21, 2022
1ad9c84
wip wip
jtkech Jun 22, 2022
c0e4597
wip
jtkech Jun 25, 2022
9fd4ec7
Testing code
jtkech Jun 26, 2022
5a56f7a
Testing code
jtkech Jun 27, 2022
66f62dd
DataMigrationAnalyzer
jtkech Jun 28, 2022
8138ced
wip
jtkech Jun 29, 2022
8889d98
Minor change
jtkech Jun 29, 2022
afa4278
Wip on removing Tenant Tables
jtkech Jun 30, 2022
1107887
Refactoring
jtkech Jul 1, 2022
8d53d84
Refactoring
jtkech Jul 2, 2022
ac08909
wip
jtkech Jul 3, 2022
2f86999
Wip on testing code
jtkech Jul 4, 2022
524e777
wip wip
jtkech Jul 5, 2022
92dcec1
wip
jtkech Jul 6, 2022
0003a3f
Minor changes
jtkech Jul 7, 2022
db7253e
wip
jtkech Jul 7, 2022
d34e44b
wip
jtkech Jul 8, 2022
cd6f194
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Jul 8, 2022
2e380a7
comment
jtkech Jul 8, 2022
7b8edae
wip
jtkech Jul 9, 2022
a328836
Use distributed lock while removing a tenant.
jtkech Jul 12, 2022
ffb9ff8
Start working on some Media Removing handlers.
jtkech Jul 13, 2022
eb31030
Wip on removing handlers
jtkech Jul 13, 2022
47876a8
Some refactoring
jtkech Jul 13, 2022
b05349a
Work on the UI
jtkech Jul 14, 2022
0911bd1
Remove testing code
jtkech Jul 14, 2022
24de85a
Remove non related component.
jtkech Jul 14, 2022
8be9ad3
UI notifications and fixes.
jtkech Jul 14, 2022
39d5007
Minor change before merging
jtkech Jul 14, 2022
d5b489c
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Jul 14, 2022
c71bae3
Revert non related changes.
jtkech Jul 14, 2022
2add2f1
Allows to remove Uninitialized Tenant.
jtkech Jul 14, 2022
f1a2645
Wip and refactoring
jtkech Jul 15, 2022
fe99d8e
Minor changes
jtkech Jul 15, 2022
9730709
Some Renaming.
jtkech Jul 15, 2022
657b4a7
Minor change
jtkech Jul 15, 2022
938b090
Minor cha,ge in view
jtkech Jul 15, 2022
de11f84
wip
jtkech Jul 16, 2022
662116d
Tweaks
jtkech Jul 25, 2022
7171ed5
Comment update
jtkech Jul 26, 2022
e372d05
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Jul 27, 2022
e5cb17a
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Aug 2, 2022
e4b409a
Minor changes
jtkech Aug 3, 2022
5131ca0
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Aug 4, 2022
2a6253d
Update after merging main
jtkech Aug 4, 2022
27c8454
Update src/OrchardCore/OrchardCore.FileStorage.AmazonS3/AwsStorageOpt…
jtkech Aug 4, 2022
f025796
Update src/OrchardCore.Modules/OrchardCore.Media.AmazonS3/CreateMedia…
jtkech Aug 4, 2022
ee018cb
Update src/OrchardCore/OrchardCore.Abstractions/Shell/Removing/IShell…
jtkech Aug 4, 2022
7c6e59a
Update src/OrchardCore/OrchardCore.Abstractions/Shell/Removing/IShell…
jtkech Aug 4, 2022
b4bbc74
Update src/OrchardCore/OrchardCore.Abstractions/Shell/Removing/IShell…
jtkech Aug 4, 2022
3466cc1
Update src/OrchardCore/OrchardCore.Abstractions/Shell/Removing/IShell…
jtkech Aug 4, 2022
b79675a
React to review
jtkech Aug 4, 2022
f908d45
Add comment on the removing ordering
jtkech Aug 5, 2022
0dce570
Minor change
jtkech Aug 5, 2022
4dd7272
Some refactoring after review
jtkech Aug 5, 2022
f875fc0
comments updates
jtkech Aug 5, 2022
2b4b560
Minor change
jtkech Aug 5, 2022
2abc21f
Useless string interpolations
jtkech Aug 5, 2022
53a7860
Usage of IStringLocalizer
jtkech Aug 5, 2022
66dae38
Fixes Distributed Tenant Removal
jtkech Aug 7, 2022
091c585
Minor change
jtkech Aug 7, 2022
0c57737
Sharing violation on removing local resources
jtkech Aug 7, 2022
30cf0ed
Adding some safeguards
jtkech Aug 7, 2022
86e1b23
Simplify Tenant Removal Syncing.
jtkech Aug 8, 2022
739cf26
Use same distributed identifier when adding or removing tenants
jtkech Aug 9, 2022
58b81a7
Keep in sync with #12174 before it gets merged.
jtkech Aug 11, 2022
73783e2
Minor tweaks
jtkech Aug 11, 2022
b21049a
Minor changes
jtkech Aug 13, 2022
7458458
Revert change managed in a separate PR
jtkech Aug 13, 2022
003152a
Renaming
jtkech Aug 14, 2022
e819a47
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Aug 15, 2022
0226c6e
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Aug 25, 2022
eafb0da
Log warning on sharing violation while removing a tenant folder.
jtkech Aug 25, 2022
1c6aaa3
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Sep 2, 2022
6fa3878
Update notifier error message
jtkech Sep 2, 2022
1865443
Add notification/logging when tenants are removed successfully
jtkech Sep 3, 2022
23fc1a4
Check LogLevel
jtkech Sep 3, 2022
9a3d7b0
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Sep 8, 2022
ff02916
Working on options
jtkech Sep 8, 2022
d44d5bb
Minor change
jtkech Sep 9, 2022
a7fcbf0
Add Tenant Disable, Enable and Remove API
jtkech Sep 10, 2022
7e1fe47
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Sep 11, 2022
7e6a68e
wip
jtkech Sep 12, 2022
a1bed5a
Cleanup Tables of an Uninitialized tenant whose setup has failed
jtkech Sep 12, 2022
1464d91
Minor tweaks
jtkech Sep 12, 2022
9a0e5e3
Minor change
jtkech Sep 12, 2022
5b69018
Minor change
jtkech Sep 12, 2022
efb38d0
Minor change
jtkech Sep 12, 2022
b020a67
Some last tweaks
jtkech Sep 13, 2022
fb21d36
Use Provider Name Constants
jtkech Sep 13, 2022
5b41f65
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Sep 14, 2022
8b831c3
Minor change
jtkech Sep 14, 2022
553573c
Minor change
jtkech Sep 16, 2022
5e23f9f
Minor changes
jtkech Sep 16, 2022
704925e
Little renaming
jtkech Sep 18, 2022
333f408
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Sep 19, 2022
1124136
Update after merging dev
jtkech Sep 19, 2022
f5c6659
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Sep 19, 2022
7a2b6ce
Fix File sharing violation
jtkech Sep 19, 2022
0a75412
Exception Extension
jtkech Sep 20, 2022
b83faf3
Minor changes
jtkech Sep 20, 2022
e5f9b76
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Sep 20, 2022
31e9fa8
Minor changes
jtkech Sep 20, 2022
6ce00db
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Oct 4, 2022
14e748b
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Oct 9, 2022
ace7af0
React to review
jtkech Oct 9, 2022
be0e8e3
Elastic also needs to support removal
jtkech Oct 10, 2022
e382f5d
Minor change
jtkech Oct 10, 2022
7f3b077
Renaming
jtkech Oct 10, 2022
0c8d2cc
Simplification
jtkech Oct 10, 2022
bb5ab09
Remove some useless try catch blocks
jtkech Oct 10, 2022
2f5ae4c
Update src/docs/reference/modules/Tenants/README.md
jtkech Oct 11, 2022
5552882
Update src/OrchardCore/OrchardCore.Search.Elasticsearch.Core/Services…
jtkech Oct 11, 2022
43ba13d
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Oct 17, 2022
715a221
Update after merging - Add CreateSchema - React to review
jtkech Oct 17, 2022
ece859d
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Oct 31, 2022
47840b6
Update after merging dev
jtkech Oct 31, 2022
85270e3
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Jan 7, 2023
235ecf5
Updates after merging dev
jtkech Jan 7, 2023
6b537d0
Update after merging dev
jtkech Jan 7, 2023
2d9243f
Merge remote-tracking branch 'origin/main' into jtkech/tenant-removal
jtkech Jan 11, 2023
71e1855
Move video to end of Tenants doc dection
jtkech Jan 11, 2023
7d0cf21
Update src/OrchardCore.Modules/OrchardCore.Tenants/Controllers/AdminC…
jtkech Jan 11, 2023
0eb6883
Update src/OrchardCore.Modules/OrchardCore.Tenants/Controllers/AdminC…
jtkech Jan 11, 2023
3bd34a9
Update src/OrchardCore.Modules/OrchardCore.Tenants/Controllers/ApiCon…
jtkech Jan 11, 2023
c26cad9
Update src/docs/reference/modules/Tenants/README.md
jtkech Jan 11, 2023
789bf47
React to reviews
jtkech Jan 11, 2023
4a37369
Hide Remove Bulk Action only from the view and '_imageSharpFolder' co…
jtkech Jan 11, 2023
e2080c6
Capture exception while creating a shellContext before a tenant remov…
jtkech Jan 12, 2023
63104e4
Remove testing config
jtkech Jan 12, 2023
8bd8859
Minor change
jtkech Jan 12, 2023
3c6caaa
Remove Bulk Action
jtkech Jan 13, 2023
6c28848
ShellSettings IsRemvable extension
jtkech Jan 13, 2023
4abfd70
Remove duplicate file
jtkech Jan 13, 2023
5120221
Update Log Messages
jtkech Jan 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions src/OrchardCore.Cms.Web/appsettings.json
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@
// },
// "BasePath": "/media",
// "CreateBucket": true,
// "RemoveBucket": true, // Whether the 'Bucket' is deleted if the tenant is removed, false by default.
// "BucketName": ""
//},
// See https://docs.orchardcore.net/en/latest/docs/reference/modules/Media.Azure/#configuration to configure media storage in Azure Blob Storage.
Expand All @@ -69,6 +70,7 @@
// "ContainerName": "somecontainer", // Set to the Azure Blob container name. Templatable, refer docs.
// "BasePath": "some/base/path", // Optionally, set to a path to store media in a subdirectory inside your container. Templatable, refer docs.
// "CreateContainer": true // Activates an event to create the container if it does not already exist.
// "RemoveContainer": true // Whether the 'Container' is deleted if the tenant is removed, false by default.
//},
// See https://stackexchange.github.io/StackExchange.Redis/Configuration.html
//"OrchardCore_Redis": {
Expand Down Expand Up @@ -235,6 +237,9 @@
//},
//"OrchardCore_Localization_CultureOptions": {
// "IgnoreSystemSettings": true
//},
//"OrchardCore_Tenants": {
// "TenantRemovalAllowed": true // Whether tenant removal is allowed, false by default.
//}
}
}
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
using System;
using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using Amazon;
using Amazon.Extensions.NETCore.Setup;
using Amazon.Runtime;
using Amazon.Runtime.CredentialManagement;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;
using OrchardCore.Environment.Shell.Configuration;
Expand Down Expand Up @@ -42,6 +40,7 @@ public static AwsStorageOptions BindConfiguration(this AwsStorageOptions options
options.BucketName = section.GetValue(nameof(options.BucketName), String.Empty);
options.BasePath = section.GetValue(nameof(options.BasePath), String.Empty);
options.CreateBucket = section.GetValue(nameof(options.CreateBucket), false);
options.RemoveBucket = section.GetValue(nameof(options.RemoveBucket), false);

try
{
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
using System;
using System.Threading.Tasks;
using Amazon.S3;
using Amazon.S3.Model;
using Amazon.S3.Util;
using Microsoft.Extensions.Localization;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using OrchardCore.Environment.Shell;
using OrchardCore.Environment.Shell.Removing;
using OrchardCore.FileStorage.AmazonS3;
using OrchardCore.Modules;

namespace OrchardCore.Media.AmazonS3;

public class MediaS3BucketTenantEvents : ModularTenantEvents
{
private readonly ShellSettings _shellSettings;
private readonly AwsStorageOptions _options;
private readonly IAmazonS3 _amazonS3Client;
private readonly IStringLocalizer S;
private readonly ILogger _logger;

public MediaS3BucketTenantEvents(
ShellSettings shellSettings,
IAmazonS3 amazonS3Client,
IOptions<AwsStorageOptions> options,
IStringLocalizer<MediaS3BucketTenantEvents> localizer,
ILogger<MediaS3BucketTenantEvents> logger)
{
_shellSettings = shellSettings;
_amazonS3Client = amazonS3Client;
_options = options.Value;
S = localizer;
_logger = logger;
}

public override async Task ActivatingAsync()
{
if (!_options.CreateBucket ||
_shellSettings.State == Environment.Shell.Models.TenantState.Uninitialized ||
String.IsNullOrEmpty(_options.BucketName))
{
return;
}

_logger.LogDebug("Testing Amazon S3 Bucket {BucketName} existence", _options.BucketName);

try
{
var bucketExists = await AmazonS3Util.DoesS3BucketExistV2Async(_amazonS3Client, _options.BucketName);
if (bucketExists)
{
_logger.LogInformation("Amazon S3 Bucket {BucketName} already exists.", _options.BucketName);

return;
}

var bucketRequest = new PutBucketRequest
{
BucketName = _options.BucketName,
UseClientRegion = true
};

// Trying to create bucket.
var response = await _amazonS3Client.PutBucketAsync(bucketRequest);

if (!response.IsSuccessful())
{
_logger.LogError("Unable to create Amazon S3 Bucket {BucketName}", _options.BucketName);

return;
}

// Blocking public access for the newly created bucket.
var blockConfiguration = new PublicAccessBlockConfiguration
{
BlockPublicAcls = true,
BlockPublicPolicy = true,
IgnorePublicAcls = true,
RestrictPublicBuckets = true
};

await _amazonS3Client.PutPublicAccessBlockAsync(new PutPublicAccessBlockRequest
{
PublicAccessBlockConfiguration = blockConfiguration,
BucketName = _options.BucketName
});

_logger.LogDebug("Amazon S3 Bucket {BucketName} created.", _options.BucketName);
}
catch (AmazonS3Exception ex)
{
_logger.LogError(ex, "Unable to create Amazon S3 Bucket.");
}
}

public override async Task RemovingAsync(ShellRemovingContext context)
{
if (!_options.RemoveBucket || String.IsNullOrEmpty(_options.BucketName))
{
return;
}

try
{
var bucketExists = await AmazonS3Util.DoesS3BucketExistV2Async(_amazonS3Client, _options.BucketName);
if (!bucketExists)
{
return;
}

var bucketRequest = new DeleteBucketRequest
{
BucketName = _options.BucketName,
UseClientRegion = true
};

// Trying to delete bucket.
var response = await _amazonS3Client.DeleteBucketAsync(bucketRequest);
if (!response.IsSuccessful())
{
_logger.LogError("Failed to remove the Amazon S3 Bucket {BucketName}", _options.BucketName);
context.ErrorMessage = S["Failed to remove the Amazon S3 Bucket '{0}'.", _options.BucketName];
}
}
catch (AmazonS3Exception ex)
{
_logger.LogError(ex, "Failed to remove the Amazon S3 Bucket {BucketName}", _options.BucketName);
context.ErrorMessage = S["Failed to remove the Amazon S3 Bucket '{0}'.", _options.BucketName];
context.Error = ex;
}
}
}
21 changes: 8 additions & 13 deletions src/OrchardCore.Modules/OrchardCore.Media.AmazonS3/Startup.cs
Original file line number Diff line number Diff line change
Expand Up @@ -69,16 +69,15 @@ public override void ConfigureServices(IServiceCollection services)
var shellSettings = serviceProvider.GetRequiredService<ShellSettings>();
var logger = serviceProvider.GetRequiredService<ILogger<DefaultMediaFileStoreCacheFileProvider>>();

var mediaCachePath = GetMediaCachePath(hostingEnvironment,
DefaultMediaFileStoreCacheFileProvider.AssetsCachePath, shellSettings);
var mediaCachePath = GetMediaCachePath(
hostingEnvironment, shellSettings, DefaultMediaFileStoreCacheFileProvider.AssetsCachePath);

if (!Directory.Exists(mediaCachePath))
{
Directory.CreateDirectory(mediaCachePath);
}

return new DefaultMediaFileStoreCacheFileProvider(logger, mediaOptions.AssetsRequestPath,
mediaCachePath);
return new DefaultMediaFileStoreCacheFileProvider(logger, mediaOptions.AssetsRequestPath, mediaCachePath);
});

// Replace the default media file provider with the media cache file provider.
Expand All @@ -103,15 +102,13 @@ public override void ConfigureServices(IServiceCollection services)
var amazonS3Client = serviceProvider.GetService<IAmazonS3>();

var fileStore = new AwsFileStore(clock, storeOptions, amazonS3Client);

var mediaUrlBase =
$"/{fileStore.Combine(shellSettings.RequestUrlPrefix, mediaOptions.AssetsRequestPath)}";
var mediaUrlBase = $"/{fileStore.Combine(shellSettings.RequestUrlPrefix, mediaOptions.AssetsRequestPath)}";

var originalPathBase = serviceProvider.GetRequiredService<IHttpContextAccessor>().HttpContext
?.Features.Get<ShellContextFeature>()
?.OriginalPathBase;

if (originalPathBase.HasValue && !String.IsNullOrWhiteSpace(originalPathBase.Value))
if (originalPathBase.HasValue)
{
mediaUrlBase = fileStore.Combine(originalPathBase.Value, mediaUrlBase);
}
Expand All @@ -126,12 +123,10 @@ public override void ConfigureServices(IServiceCollection services)

services.AddSingleton<IMediaEventHandler, DefaultMediaFileStoreCacheEventHandler>();

services.AddScoped<IModularTenantEvents, CreateMediaS3BucketEvent>();
services.AddScoped<IModularTenantEvents, MediaS3BucketTenantEvents>();
}
}

private string GetMediaCachePath(IWebHostEnvironment hostingEnvironment,
string assetsPath, ShellSettings shellSettings)
=> PathExtensions.Combine(hostingEnvironment.WebRootPath,
assetsPath, shellSettings.Name);
private static string GetMediaCachePath(IWebHostEnvironment hostingEnvironment, ShellSettings shellSettings, string assetsPath)
=> PathExtensions.Combine(hostingEnvironment.WebRootPath, shellSettings.Name, assetsPath);
}

This file was deleted.

Loading