Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore conflicting concurrent storage writes of the same data with version="*" #1118

Closed
wants to merge 6 commits into from

Conversation

ftkg
Copy link
Contributor

@ftkg ftkg commented Oct 25, 2023

Upsert with DO NOTHING will ignore insert on conflicting key, instead of failing. The intended upside is that transactions with conflicting inserts will not have to be retried anymore, saving unnecessary database processing.

Results do not change, as the new query will still return no rows on conflict, hence the batch insert will be aborted with ErrStorageRejectedVersion.

@ftkg ftkg requested a review from redbaron October 25, 2023 14:44
server/core_storage.go Outdated Show resolved Hide resolved
@ftkg ftkg requested a review from redbaron October 25, 2023 21:31
@ftkg ftkg changed the title Ignore conflicting storage inserts with version="*" [WIP] Ignore conflicting storage inserts with version="*" Oct 25, 2023
@ftkg
Copy link
Contributor Author

ftkg commented Oct 26, 2023

New query should return success for concurrent writes of identical objects. Had to use ON CONFLICT UPDATE to return a row for this scenario, that's an extra step but it should be better than retrying (additionally, the previous WHERE NOT EXISTS clause will avoid unnecessary updates for non-concurrent duplicate writes and keep the semantics of rejecting them).

@redbaron
Copy link
Contributor

redbaron commented Oct 26, 2023

UNION ALL LIMIT 1 variant doesn't unnecessary overwrite row just to return it and can be made to work by checking returned row on the Go side: If returned row is the same which is written, then write succeeds.

I see what you mean by "concurrent safe"

@ftkg ftkg changed the title [WIP] Ignore conflicting storage inserts with version="*" Ignore conflicting concurrent storage inserts with version="*" Oct 27, 2023
@ftkg ftkg changed the title Ignore conflicting concurrent storage inserts with version="*" Ignore conflicting concurrent storage writes of the same data with version="*" Oct 27, 2023
@redbaron
Copy link
Contributor

This is a behaviour change which needs to be prominently mentioned in release notes and docs. Previously it was possible to use this operation to create distributed lock/ leader election , now it is still possible but written value must be unique for each node

@zyro
Copy link
Member

zyro commented Jul 10, 2024

After internal discussion, we determined the current behaviour is actually desirable: "*" must fail on any existing value even if fully identical. This allows Nakama to support use cases such as using a storage object as a lock or signal if needed.

We may however introduce a new OCC operator at some point that allows the write to succeed if the object already existed but the value was identical.

@zyro zyro closed this Jul 10, 2024
@zyro zyro deleted the ft-adjust-storage-star-write branch July 10, 2024 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants