Skip to content
This repository has been archived by the owner on Jul 7, 2024. It is now read-only.

Woodstar experiment summary #22

Closed
roji opened this issue Jul 1, 2024 · 0 comments
Closed

Woodstar experiment summary #22

roji opened this issue Jul 1, 2024 · 0 comments

Comments

@roji
Copy link
Member

roji commented Jul 1, 2024

First, thank you for being so patient waiting for news here; we probably should have provided more timely updated on the project state. This was partially a result of us simply not knowing what's going to happen with Woodstar and SqlClient (e.g. the recently-started SqlClientX - see below), and partially a result of us simply being very overloaded with other things.

Woodstar's original idea was an exploratory, greenfield SQL Server (TDS) driver; the goal was to use modern, high-performance .NET techniques, liberated from SqlClient's technical debt, and to see where that would lead in terms of performance. Specifically, we were interested in seeing what kind of performance gains we would see on the TechEmpower Fortunes benchmark, compared to SqlClient. There was no clear future for Woodstar as an actually supported product that's usable in production - it was purely a technical experiment.

The main work actually done was initial experimentation/prototyping by @NinoFloris (a core contributor on Npgsql) and myself; we built a minimal TDS client that could support TechEmpower Fortunes, and nothing more; for example, parameters were not yet supported, as well as many other features. The experiment was async-only, did not implement ADO.NET, and used System.IO.Pipelines for I/O. The prototype source code is available on this repo, as-is: it really is just an exploratory prototype, nothing more.

For the very simple TechEmpower Fortunes scenario, the prototype did not provide meaningful performance improvements over SqlClient. This does not mean that SqlClient has no performance issues: it certainly does (see this discussion) - just not in the very narrow TechEmpower Fortunes usage scenario. Our exploration did yield some valuable conclusions; two important ones are the following:

  • We gained some interesting insights around TDS and its processing that impact both the client and the server side, and so we engaged internally with the SQL Server org. This has been a positive engagement and various things are happening behind the scenes.
  • System.IO.Pipelines (with SequenceReader) work great when parsing relatively large payloads; but in a client-side database driver scenario, the user repeatedly calls in to parse very small values in the resultset (e.g. an int). In that kind of usage, reinstantiating a SequenceReader (ref struct) each time is too much overhead. Similarly, continuously slicing ReadOnlySequence for each tiny was costly, so there was no good way for us to cheaply store the current position.

Further work on Woodstar did not continue, simply because we had other, more important things that got prioritized over this. However, the lessons learned from the experiment were quite valuable, shared with relevant parties internally, and are present in discussions with the SqlClient team.

On the SqlClient side, the SqlClientX effort has recently begun - this is a project to reimplement the I/O layer and pooling implementation inside SqlClient, allowing users to opt into the new experimental implementation and eventually switching to it as the default. In a way, SqlClientX is the spiritual successor to Woodstar; although SqlClientX it's not a greenfield new driver since it's being done within SqlClient and must respect backwards compat, the goals of the two projects are the same - arrive at a modern, efficient SQL Server driver without all the technical debt, and which is able to evolve safely and quickly. The future of SqlClientX is also much clearer, being owned and maintained by the SqlClient itself, whereas Woodstar was purely an experiment with no clear path to becoming a supported product at any point.

For now, we will be archiving this GitHub repo, as work in this area is not happening here.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant