Implement distributed/promoted transactions in System.Transactions for Windows only

[StephenBonikowsky]

@jimcarley commented on Wed Nov 09 2016

---

@XieJJ99 commented on Tue Jun 11 2019

@ajcvickers @stephentoub The SQL Server team had already provided MSDTC for SQL Server on Linux. Is there any plan to promote the MSDTC for linux as a generic service so that it's possible to support distributed transaction on Linux?

---

@ULSTechology commented on Wed Jun 26 2019

A lack of support for distributed transactions is pretty disastrous for anyone wanting to migrate their old system onto .NET Core.

This puts an end to our hopes that we can start using Blazor - we can't rewrite our entire application to work without distributed transactions, there's just too much of it.

---

@popcatalin81 commented on Wed Jun 26 2019

we can't rewrite our entire application to work without transactions, there's just too much of it.

Surely, if you can rewrite for Blazor, which is a significant rewrite anyway, you can also eliminate distributed transactions.

---

@ULSTechology commented on Wed Jun 26 2019

we can't rewrite our entire application to work without transactions, there's just too much of it.

Surely, if you can rewrite for Blazor, which is a significant rewrite anyway, you can also eliminate distributed transactions.

Using Blazor requires a rewrite of the UI, and we can start small with that.

Rewriting some fairly large chunks of our data access layer, and probably even some of our business layer, is a whole other matter. Unfortunately, it's not an option we could consider at the moment.

---

@mmaderic commented on Thu Jul 18 2019

I would like to vote up for this feature at least for databases on single SQL server instance.

I have case where I would like to enclose two different database commits into single system transaction. (Sort of micro-service architecture).

---

@abbotware commented on Mon Jul 29 2019

@LaterStart - I have the same use case so +1

Creating multiple exe microservices (service fabric) sharing a single SQL database (migration of legacy code / data)

I found it is possible on Net Core to share transactions/connections between ADO. and EF Core within a single process (https://docs.microsoft.com/en-us/ef/core/saving/transactions) ,

However, what options currently exist when 2 separate Net Core exes are involved for implementing a 2 phase-commit like pattern?

1. Can the DTC / Resource Manager be simulated via base classes in Net Standard? (https://docs.microsoft.com/en-us/previous-versions/ms229975(v=vs.90))

2. Call the DTC directly via Win32 Api and some how enlist?

1-2 seem like a lot of work, but from what I have read (somewhere) I could just target Net Standard 2.0 and run within a Net Framework host - That will allow System.Transactions to enlist / promote into a DTC transaction, (I can settle for Netstandard + 4.7.2 + new sdk project format if that is my only option)

Even if I use a Net Framework Host (or Net Core if this open issue ever gets implemented), does that mean it should it is possible to obtain the transaction token / correlation id from process A so it can be sent/serialized to process B for manual enlistment?

---

@pappasa commented on Tue Sep 03 2019

How do you use .NET Core with IBM WebSphere MQ queue? For example, I do not want to remove the message from the queue, and the put to the database to fail.

---

@aajmot commented on Fri Sep 20 2019

can someone tell me if the issue has been fix or still there?
It's almost 2020 now?
Is there any other way around, please tell me.

---

@pi3k14 commented on Fri Oct 11 2019

@dasetser
wtf - you moved this into the unforeseeable future?
Don't you want to on-board enterprise solutions to .Net Core, you want to push us all over to the dark side (java)?

---

@am11 commented on Fri Oct 11 2019

Going by https://referencesource.microsoft.com/#System.Transactions/System/Transactions/Oletx/DTCTransactionManager.cs,2c1154236b6b7acf, seems like there are some win32 specific components involved which probably is the main hurdle. Could someone shed light on what are those and what would require to bring them cross platform?

---

@dasetser commented on Tue Oct 15 2019

This got moved to future because after investigating it we determined it's going to be a significant rewrite of the feature rather than just a port. We're looking into options for this, but we aren't able to commit to getting this done in the upcoming release.

The main hurdle is that System.Transactions depends on MSDTC when it has to promote transactions, but MSDTC is only available on Windows. In order to get this to work cross platform we would need to rewrite the feature to allow it to work with other distributed transaction coordinators that are available on other platforms, or look into doing something similar to what the SQL Server team did where they brought the DTC components inside the SQL component. Either approach would be a significant change to this feature.

---

@Peperud commented on Tue Oct 15 2019

Where do I vote for this?

---

@pi3k14 commented on Tue Oct 15 2019

@dasetser - I appreciate your clarification.
But, us current users of Distributed transactions don't care about cross platform.
Just support it on Windows, and put this cross platform paradigm that we haven't request into the future.

Thank you.

---

@am11 commented on Tue Oct 15 2019

Thanks for the explanation @dasetser.
If the target linux machine has MSDTC component from SQL Server installed, and its C API is the same as Windows, can the implementation take advantage -- as in, SQL Server as a prerequisite (for now)? We can separately request and poll SQL Server team to ship MSDTC component as a standalone library. :)

---

@randyburden commented on Thu Dec 05 2019

We were planning to begin porting many of our current applications to .NET Core starting next week as well as start writing our new apps on .NET Core but we just ran into this issue of distributed transactions not working which feels like a deal breaker. Almost every application we write or will write in the future requires distributed transactions.

It is very odd to have experienced a platform not supported exception with this particular feature when we all know that it already exists in the full .NET Framework. It's hard to understand why a Windows platform specific implementation wasn't implemented either in CoreFX or as a shim NuGet package.

Is there a workaround?

---

@pi3k14 commented on Fri Dec 06 2019

@dasetser - Wouldn't implementing WS-AtomicTransaction be the way to go. And since MSDTC supports it and WCF has an implementation of it https://docs.microsoft.com/en-us/dotnet/framework/wcf/feature-details/using-ws-atomictransaction this shouldn't take that much time. Those wanting distributed transactions on Unix can then get a third party solution.

[iCodeWebApps]

I am happy that the resolve to this issue has been added to the 5.0 Milestone. Cheers!

[turabek]

For Enterprise applications it is very important to have a distributed transaction support. Happy to see this feature as part of milestone 5.

[pappasa]

Do you mean DTC will be supported in .NET 5?

[RobThaBlob]

I'd like to add my voice to those requesting this feature. It seems a backward step not to support distributed transactions.

[xuyichenklkkop]

this's is why we don't use .NET Core any more.

[NullEntity]

This is a big issue for my team too. We have a large net fx system in the fintech space. We use distributed transactions pretty heavily to guarantee atomicity. What I don't like about them is that it's really easy to accidently promote a TransactionScope which makes it incredibly hard to audit. We've been slowly replacing them with an outbox implementation (they're mostly used to atomically process a message from a queue and update the db), but it's a huge amount of work.

[jimmyzimms]

While I can appreciate the fact that not every platform has a transaction manager that works in a compatible enough methodology to the Net Framework implemtation, I find it gobsmacking that years on there's still not an on-Windows implementation as a package for this. There's nothing preventing an adapter pattern to be used with the default implementation being the current, NotSupportedException, approach and allowing 3rd party packages to pick things up.
About every 6 months we pop over and check on this issue and until it's dealt with, NetCore is a deal breaker. Unfortunate.

[NullEntity]

@dasetser is this still on the roadmap for .NET 5? I've been looking forward to this functionality, but haven't heard about any updates to it in the previews.

[precision-sean]

Can we receive an update on this? Knowing whether or not this is still on the roadmap for .NET 5 has a significant impact on my company's plan to migrate from .NET Framework to .NET Core.

Thank you.

[NullEntity]

Can we receive an update on this? Knowing whether or not this is still on the roadmap for .NET 5 has a significant impact on my company's plan to migrate from .NET Framework to .NET Core.

Thank you.

I expect it to not happen because it's so late in the 5.0 release cycle and there's been no news yet. I recommend working to move away from distributed transactions.

[precision-sean]

Thank you, @HongGit, for the milestone confirmation.

[rizi]

@HongGit it seems distributed/promoted transactions still not working with .net 5.0 RC2, any time line for this?
br

[imilchev]

I am also interested in the roadmap for this. At least some indication would be useful

[jimmyzimms]

Now that we went GA on the Net 5 release I've confirmed that they still have yet to tackle this with a simple console app opening two connections in a transactionscope. [slow clap]

The lack of a pluggable middleware component into the transaction manager with a windows specific nuget package enlisting DTC is criminal at this point.

[cyanite]

I am wondering if there is any timeline update on this? Specifically I mean: How likely is it looking to make it into .NET 7 final?

[roji]

@cyanite this is one of the high-priority items on my list fo .NET 7, so I'd say there's a good chance. But as there are a lot of unknowns around this and many other tasks, it's always possible it would be punted out of the release.

[swh-cb]

For those who want to run MSDTC distributed transactions already under .NET 6.0 with multiple Microsoft SQL Servers on the Windows platform, our library might be of interest:

https://github.com/softwarehelden-com/Softwarehelden.Transactions.Oletx

The library is now open source. We have been using the library internally in our deployment processes for a few months now as a workaround to run distributed transactions on multiple MSSQL servers. At runtime we still use .NET Framework 4.8 because we need to support distributed transactions with Oracle database servers as well. Please read the readme in the repo to learn how it works under the hood.

Update (Version 1.1.0): Applications targeting .NET Core 3.1 or .NET 5.0 can now use the library to perform distributed transactions for MSSQL servers.

Update (Version 1.2.0): It is now possible to enlist Oracle database servers and MSSQL servers in the same distributed MSDTC transaction. This means we can finally target .NET 6.0 for our runtime environments. :-)

Update (Version 1.3.0): Data providers can now be loaded in an isolated compatibility assembly load context if they are not 100% compatible with .NET Core. For example, the unmanaged ODP.NET provider uses types and methods from the .NET Framework unknown to the .NET Core runtime to coordinate the distributed transaction between MSDTC and Oracle.

[yedf2]

Here is a commonly used distributed transaction framework which also support dotnet.
Transaction coordinator: https://github.com/dtm-labs/dtm
Dotnet SDK: https://github.com/dtm-labs/dtmcli-csharp
A blog about how to design a distributed transaction: https://www.c-sharpcorner.com/article/distributed-transaction-in-c-sharp-microservices-using-saga-pattern/

[roji]

FYI work is ongoing to port the distributed transaction support from .NET Framework, to .NET 7.0. Unfortunately, that support is tightly coupled to the Windows MSDTC service (and the OleTx protocol), so this will work on Windows only (on other platforms, starting a distributed transaction will throw as today). Enabling distributed transactions in a cross-platform way is a significantly bigger task, and is now being tracked in #71769.

Since it's late in the .NET 7.0 release, we're not fully sure that work will be done in time - but we're doing the best we can. Note that some specific functionality that worked in .NET Framework may be excluded from this port (we'll provide more details later).

[rwasef1830]

@roji Is there any prominent possible alternative 3rd party library you are aware of that cross platform can use in the meantime, and is there an integration point for such API into the .NET API ? such as a provider model ? or any pluggable mechanism ?

[roji]

@rwasef1830 not to my knowledge.

The problem with 2PC is that it requires coordinated support across a transaction manager (currently MSDTC only), the multiple database drivers out there, and a user-facing API for managing the transaction (currently System.Transactions). Because of this I don't think it's possible for some 3rd-party library to just "handle" all of it; at minimum, database drivers which are currently written against the Sys.Tx support would have to be adjusted to work against a another more cross-platform one.

I've spent some time researching what this would would mean and how it would look like; it's not trivial, and we're not sure we're seeing lots of user demand for it (#71769 is for gauging that). In the meantime, see my comment for alternatives to 2PC which may be more appropriate in modern cloud settings.

[SimonCropp]

@roji awesome. thanks