Region failover strategy

[bar10dr]

Basically if region 1 goes down, we would like region 2 to take over the traffic. Which means data has to be replicated in realtime between the two regions.

Is it recommended to use ZoneTree if you have regional failover needs? If so, how would you go about implementing that?

First thought was have one main location for writes and then use something like inotify to sync files to a secondary location, then use DNS faileover as a method to transfer write rights to the secondary if the first location goes down.

A problem would pop up in case of a netsplit, where some nodes are connected to location 1 and other nodes are connected to location 2, where suddenly both data providers think they are the master.
How do you deal with the sync again after everything comes back up again?

Usually this problem overcome with having three or more (Or an odd number perhaps) independent providers, where you need the consensus of the majority interconnected provider count in order to claim who is the master, with the lesser node count network realizing they do not have enough consensus to keep the crown; in this usecase though with koculu it would require a lot more than just DNS failover to achieve.

Any strategy recomandations? Any third party solutions that work with file storage systems like ZoneTree out there that already solves this?

Or am I simply looking at the wrong tool for the job?

[bar10dr]

Having looked more closely at the problem, I assume you would have to implement some sort of raft solution, have writes happen on the main and simultaniously send the data to the read only nodes.

If split, raft decides what any given location service can and cannot do.

Then you have to manually handle the data sync after netsplit is over.

https://dotnet.github.io/dotNext/features/cluster/raft.html

[koculu]

ZoneTree does not provide a consensus algorithm, and you can use any algorithm that you wish.

ZoneTree provides opIndex to correctly order outgoing writes asynchronously. Depending on your algorithm choice, you may need the opIndex.
A simple Replicator class that replicates data to a secondary location is also in the repository:

https://github.com/koculu/ZoneTree/blob/main/src/ZoneTree/Core/Replicator.cs
https://github.com/koculu/ZoneTree/blob/main/src/ZoneTree.UnitTests/ReplicatorTests.cs