Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Concurrently adding new fields from different processes is unstable #3185

Closed
brimoor opened this issue Jun 12, 2023 · 0 comments · Fixed by #3308
Closed

[BUG] Concurrently adding new fields from different processes is unstable #3185

brimoor opened this issue Jun 12, 2023 · 0 comments · Fixed by #3308
Labels
bug Bug fixes

Comments

@brimoor
Copy link
Contributor

brimoor commented Jun 12, 2023

Executing the steps below in the order indicated raises errors on fiftyone<0.22.

The root problem is multiple processes adding elements to DatasetDocument.sample_fields concurrently. Rather than both adding elements using array updates, they are both be $set-ing the entire field, as mongoengine does not support intelligent granular updates of array fields.

TERMINAL A

import fiftyone as fo
import fiftyone.zoo as foz

# step 1
dataset = foz.load_zoo_dataset("quickstart", dataset_name="zzz")
dataset.persistent = True

# step 3
sample = dataset.first()
sample["foo"] = "bar"

# step 5
sample.save()

# step 7: ERROR
dataset.reload()
# FieldDoesNotExist: The fields "{'spam'}" do not exist on the document "samples.XXXXXXXX"

TERMINAL B

import fiftyone as fo

# step 2
dataset = fo.load_dataset("zzz")

# step 4
sample = dataset.first()
sample["spam"] = "eggs"

# step 6
sample.save()

# step 8: ERROR
dataset.reload()
# FieldDoesNotExist: The fields "{'foo'}" do not exist on the document "samples.XXXXXXXX"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug fixes
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant