Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Concurrently updating list field in a sample from different processes does not keep all changes #3187

Open
brimoor opened this issue Jun 12, 2023 · 0 comments
Labels
bug Bug fixes core Issues related to Core features

Comments

@brimoor
Copy link
Contributor

brimoor commented Jun 12, 2023

Due to the same underlying reason explained in #3185, concurrently updating a list field of a sample held in-memory in multiple processes can cause data loss.

Note that this is only an issue if a sample object is held in-memory in one process while being edited in another process, which is not as common as #3185.

Ideally this would be fixed by Document._save() intelligently performing array updates using $push, $pull, etc rather than $set-ing the entire field:

sets, unsets = self._delta()

But unfortunately mongoengine does not natively support this:
https://docs.mongoengine.org/guide/document-instances.html#saving-and-deleting-documents

TERMINAL A

import fiftyone as fo
import fiftyone.zoo as foz

# step 1
ds = foz.load_zoo_dataset('quickstart', max_samples=2)
ds.persistent = True

# step 2
s = ds.first()
s['listfld'] = ['a']
s.save()

# step 5
s['listfld'] += ['c','cc']

# step 7
s.save()

# step 9
ds.reload()

print(ds.first())
# Output: listfld contains ['a','c','cc']

TERMINAL B

import fiftyone as fo

# step 3
ds = fo.load_dataset('quickstart-2')

# step 4
s = ds.first()
s['listfld'] += ['b','bb']

# step 6
s.save()

# step 8
ds.reload()

print(ds.first())
# Output: listfld contains ['a','c','cc']
@brimoor brimoor added bug Bug fixes core Issues related to Core features labels Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug fixes core Issues related to Core features
Projects
None yet
Development

No branches or pull requests

1 participant