Migrate event sourced data from one tenant to another

[WilvanBil]

I work for a company FooBar. FooBar has 3 departments

• Department1
• Department2
• Department3

They all have to create Foo. Foo is a SingleStreamAggregation that is multitenanted. It has the usual events:

• FooCreatedDomainEvent
• FooUpdatedDomainEvent
• Foo....

All is good, departments can't view each other's Foo data, but there's still the possibility for certain cross tenant business logic.
Now the higherups have decided they need to get rid of Department1. They want to fuse Department1 with Department3.
This means that all Foo data needs to be transferred to Department3 with the history of events being intact.

I would like to suggest the following:

        await using var session = documentStore.LightweightSession(Tenants.Department1);
        await session.Events.MigrateStreamAsync<Foo>(fooToMigrate.Id, Tenants.Department3, token: cancellationToken);

Perhaps even provide a bulk action method?
This method will take the events and stream records of streamid fooToMigrate.Id, migrate it to the new tenant and based on projection type maybe even rebuild the Foo document projection of fooToMigrate.Id.

There's ofcourse still the issue of the event being used in other projections and what to do with those.
Imho I think it's a business decision to make sure these projections are also migrated to the correct tenant.
It would also be nice to link this to an event for example a FooMigratedDomainEvent: IMigrationTrigger, this way we can also add it as an event to the stream itself.

I would like to start a discussion regarding the possible issues and solutions to this. I think it's a relevant enough feature that may occur in real life.
The following is a hacky DO ON YOUR OWN RISK work around on migrating streams to another tenant. Only for TenancyStyle.Conjoined
YOU HAVE BEEN WARNED!

-- Make sure application is not running so no database manipulations are happening
-- Begin illegal migration technique
-- Drop the foreign key constraint on mt_events
ALTER TABLE public.mt_events
DROP CONSTRAINT IF EXISTS fkey_mt_events_stream_id_tenant_id;

-- Update tenant_id in mt_events
BEGIN;
UPDATE public.mt_events
SET tenant_id = 'Department3'
WHERE tenant_id = 'Department1';
commit;

-- Update tenant_id in mt_streams
BEGIN;
UPDATE public.mt_streams ms
SET tenant_id = 'Department3'
WHERE tenant_id = 'Department1'
COMMIT;

-- Update tenant_id in mt_doc_foo
BEGIN
UPDATE public.mt_doc_foo
SET tenant_id = 'Department3',
data = jsonb_set(data, '{TenantId}', '"Department3"') -- only if TenantId is a property on your projection
WHERE tenant_id = 'Department1';
COMMIT;

-- Add the constraint
ALTER TABLE public.mt_events
ADD CONSTRAINT fkey_mt_events_stream_id_tenant_id FOREIGN KEY(stream_id, tenant_id)
REFERENCES public.mt_streams(id, tenant_id);
-- Act like nothing has happened

[iamjustsam]

The proposed solution might work for smaller datasets but if you're migrating lots of streams this might not be the ideal approach.
Another solution might be to have an API where you can define tenant mappings so Marten can generate tenant_id filters transparently.

Something like this:

storeOptions.Events.MergeTenants("SourceTenantId", "DestinationTenantId");

The resulting WHERE clause for any query to either tenant would be:

WHERE tenant_id IN ('SourceTenantId', 'DestinationTenantId')

[WilvanBil]

I did not even think about bigger datasets, you are right. It is not a fast strategy to loop over the streams.
What I understand from your approach is that the tenant will remain the same, but in lookup it will add a behind the scenes contains in tenants(...). This seems like a very reasonable approach for the merging of multiple tenants into one, regardless of source so I like this approach.

How would Marten store the information from the MergeTenants? Would it be an extra table where you can store the mappings?
Also what if it's not a Conjoined style tenancy, but with different databases? Maybe store it on each database but duplicate the tables?

I'm currently also thinking about a department that's going to be split into different departments. So instead of merging, we need a split strategy :D. But that's an entire different can of worms and not relevant to this discussion.