vitessdriver: add support for DistributedTx #9451
Conversation
Signed-off-by: Derek Perkins <derek@nozzle.io>
derekperkins
added
Type: Enhancement
Signed-off-by: Derek Perkins <derek@nozzle.io>
Signed-off-by: Derek Perkins <derek@nozzle.io>
derekperkins
force-pushed
the
distributed-tx-go-driver
branch
from
7243e1c
to
40e8160
Compare
derekperkins
requested review from
frouioui,
harshit-gangal and
systay
as code owners
this simplifies the session token usage so that the configuration object no longer has to store the session struct itself Signed-off-by: Derek Perkins <derek@nozzle.io>
Signed-off-by: Derek Perkins <derek@nozzle.io>
derekperkins
added
the
release notes (needs details)
Signed-off-by: Derek Perkins <derek@nozzle.io>
| // this is designed to be run after all new work has been done in the tx, similar to | ||
| // where you would traditionally run a tx.Commit, to help prevent you from silently | ||
| // losing transactional data. | ||
| validationFunc := func() error { | ||
| var sessionToken string | ||
| sessionToken, err = SessionTokenFromTx(ctx, tx) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| session, err = sessionTokenToSession(sessionToken) | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| if len(session.ShardSessions) > originalShardSessionCount { | ||
| return fmt.Errorf("mismatched ShardSession count: originally %d, now %d", | ||
| originalShardSessionCount, len(session.ShardSessions), | ||
| ) | ||
| } | ||
|
|
||
| return nil | ||
| } |
There was a problem hiding this comment.
I tested this in our setup and found this pathological edge case where there were no errors, but transactional data just disappeared. After tracking it down, I hadn't done any transactional work before sending over the session token, so it had no shard sessions, leaving the new ones orphaned.
The idea is that this function will help to prevent that from happening. I return this func rather than making a separate call, so that users are forced to be aware of it. The pattern is similar to a context cancel function that is returned on any new context operations, so it should feel familiar to people.
I thought about doing a deep comparison on the shard sessions, but that seemed like more work than just comparing the counts without adding any extra benefits.
without this check, database calls could appear to succeed with no errors, but the transaction ends up abandoned, potentially leaving users in an inconsistent state Signed-off-by: Derek Perkins <derek@nozzle.io>
derekperkins
force-pushed
the
distributed-tx-go-driver
branch
from
7b6532b
to
38d84a1
Compare
|
This is working well in our infrastructure, not at particularly high scale, but enough to feel good about it. I'm very glad that the first case I tried it on exposed the shard session bug - I believe this should be fairly safe for general consumption with the safeguards in place. I'll go ahead and merge with the approvals given. |
Description
This enables users of the Go driver to serialize a session and connect to an existing transaction while still using the
database/sqldriver, as presented by @dkhenry herehttps://www.cncf.io/online-programs/transactional-microservices-with-vitess-coordination-without-state/
SessionTokenis introduced, a protobuf encodedvtgatepb.Sessionrepresented as base64SessionToken:vt_session_token. This is intercepted at the Go driver layer, and is necessary to get around Go interface implementation details. The query string is not exposed to users, so it could be anything if we want. We could promote this to a vtgate construct to simplify the use case across multiple clients if we wanted to formalize the token.Txobject is prevented from callingCommitorRollback, not for technical reasons, but to try to prevent misuse. This could change in the future if we wanted, but I prefer to be more strict up front.fakeserver, as there are no current driver tests to do this end to end.Checklist