Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to RETURNING/OUTPUT as the default way to retrieve generated values for INSERT #27503

Closed
Tracked by #26797
roji opened this issue Feb 24, 2022 · 0 comments · Fixed by #27573
Closed
Tracked by #26797

Switch to RETURNING/OUTPUT as the default way to retrieve generated values for INSERT #27503

roji opened this issue Feb 24, 2022 · 0 comments · Fixed by #27573
Assignees
Labels
area-perf area-save-changes closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. customer-reported providers-beware type-enhancement
Milestone

Comments

@roji
Copy link
Member

roji commented Feb 24, 2022

#27372 is about optimizing the SQL generated for fetching generated values on SQL Server; as part of that, we will be switching to INSERT INTO foo (bar) OUTPUT VALUES ... by default, which works considerably faster than the current method.

SQL Server, PostgreSQL, MySQL and SQLite all support such a method of retrieving generated values (the standard clause is RETURNING except for in SQL Server). Aside from two SQL statements being generally more efficient (and safer) than one, two SQL statements also need to be wrapped in a transaction, to prevent a race condition where a separate update could cause different values to be returned (see #27446). With #27439 removing transactions for single updates, switching to RETURNING would unlock that perf benefit as well.

For example, SQLite shows the following improvement when switch from our current method to RETURNING:

Method Mean Error StdDev Ratio
Insert_and_Select_with_transaction 10.555 us 0.1769 us 0.1568 us 1.00
Insert_and_Select_without_transaction 5.428 us 0.0558 us 0.0495 us 0.51
Insert_Returning 5.914 us 0.0596 us 0.0497 us 0.56
Benchmark code
BenchmarkRunner.Run<Benchmark>();

public class Benchmark
{
    private SqliteConnection _connection;
    private SqliteCommand _command;

    private async Task Setup()
    {
        _connection = new SqliteConnection("Filename=:memory:");
        _connection.Open();

        using var cmd = new SqliteCommand("CREATE TABLE data (id INTEGER PRIMARY KEY AUTOINCREMENT, num INTEGER)", _connection);
        await cmd.ExecuteNonQueryAsync();
    }

    [GlobalSetup(Targets = new[] { nameof(Insert_and_Select_with_transaction), nameof(Insert_and_Select_without_transaction) })]
    public async Task Setup_Insert_and_Select_with_transaction()
    {
        await Setup();

        var sql = @"
INSERT INTO data (num) VALUES (@p0);
SELECT id FROM data WHERE changes() = 1 AND rowid = last_insert_rowid();";

        _command = new SqliteCommand(sql, _connection);
        _command.Parameters.AddWithValue("p0", 8);
    }

    [GlobalSetup(Target = nameof(Insert_Returning))]
    public async Task Setup_Insert_Returning()
    {
        await Setup();

        var sql = "INSERT INTO data (num) VALUES (@p0) RETURNING id";
        _command = new SqliteCommand(sql, _connection);
        _command.Parameters.AddWithValue("p0", 8);
    }

    [Benchmark(Baseline = true)]
    public void Insert_and_Select_with_transaction()
    {
        var tx = _connection.BeginTransaction();
        _command.Transaction = tx;

        using var reader = _command.ExecuteReader();
        reader.Read();
        for (var i = 0; i < reader.FieldCount; i++)
            _ = reader.GetInt32(i);

        tx.Commit();
    }

    [Benchmark]
    public void Insert_and_Select_without_transaction()
    {
        using var reader = _command.ExecuteReader();
        reader.Read();
        for (var i = 0; i < reader.FieldCount; i++)
            _ = reader.GetInt32(i);
    }

    [Benchmark]
    public void Insert_Returning()
    {
        using var reader = _command.ExecuteReader();
        reader.Read();
        for (var i = 0; i < reader.FieldCount; i++)
            _ = reader.GetInt32(i);
    }
}

/cc @lauxjpn

@roji roji self-assigned this Feb 24, 2022
@ajcvickers ajcvickers added this to the 7.0.0 milestone Mar 1, 2022
roji added a commit to roji/efcore that referenced this issue Mar 4, 2022
…tions

Also make RETURNING the default INSERT strategy for retrieving
db-generated values (for other providers).

Fixes dotnet#27372
Fixes dotnet#27503
roji added a commit to roji/efcore that referenced this issue Mar 4, 2022
…tions

Also make RETURNING the default INSERT strategy for retrieving
db-generated values (for other providers).

Fixes dotnet#27372
Fixes dotnet#27503
roji added a commit to roji/efcore that referenced this issue Mar 4, 2022
…tions

Also make RETURNING the default INSERT strategy for retrieving
db-generated values (for other providers).

Fixes dotnet#27372
Fixes dotnet#27503
roji added a commit to roji/efcore that referenced this issue Mar 9, 2022
…tions

Also make RETURNING the default INSERT strategy for retrieving
db-generated values (for other providers).

Fixes dotnet#27372
Fixes dotnet#27503
@roji roji added the closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. label Mar 10, 2022
roji added a commit to roji/efcore that referenced this issue Mar 16, 2022
…tions

Also make RETURNING the default INSERT strategy for retrieving
db-generated values (for other providers).

Fixes dotnet#27372
Fixes dotnet#27503
roji added a commit to roji/efcore that referenced this issue Mar 16, 2022
…tions

Also make RETURNING the default INSERT strategy for retrieving
db-generated values (for other providers).

Fixes dotnet#27372
Fixes dotnet#27503
roji added a commit to roji/efcore that referenced this issue Mar 16, 2022
…tions

Also make RETURNING the default INSERT strategy for retrieving
db-generated values (for other providers).

Fixes dotnet#27372
Fixes dotnet#27503
roji added a commit to roji/efcore that referenced this issue Mar 16, 2022
…tions

Also make RETURNING the default INSERT strategy for retrieving
db-generated values (for other providers).

Fixes dotnet#27372
Fixes dotnet#27503
roji added a commit to roji/efcore that referenced this issue Mar 17, 2022
…tions

Also make RETURNING the default INSERT strategy for retrieving
db-generated values (for other providers).

Fixes dotnet#27372
Fixes dotnet#27503
@ghost ghost closed this as completed in #27573 Mar 17, 2022
ghost pushed a commit that referenced this issue Mar 17, 2022
…tions (#27573)

Also make RETURNING the default INSERT strategy for retrieving
db-generated values (for other providers).

Fixes #27372
Fixes #27503
@roji roji changed the title Switch to RETURNING/OUTPUT as the default way to retrieve generated values Switch to RETURNING/OUTPUT as the default way to retrieve generated values for INSERT Mar 17, 2022
@ajcvickers ajcvickers modified the milestones: 7.0.0, 7.0.0-preview3 Mar 31, 2022
@ajcvickers ajcvickers modified the milestones: 7.0.0-preview3, 7.0.0 Nov 5, 2022
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-perf area-save-changes closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. customer-reported providers-beware type-enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants