Incomplete data

[mojtaba-komeili]

Patch description
Skipping units that fail under internal assertions in Mephisto and generating a warning that tells user we are skipping them. This is one issue that one might run into while dealing with corrupted data in Mephisto. Skipping them avoids a potential crash in data compiler/

Testing steps
Using it to compile my dataset. It skipped the corrupted unit as expected.

[JackUrb]

Out of curiosity, what is the AssertionError that throws here?

[mojtaba-komeili]

Out of curiosity, what is the AssertionError that throws here?

This is what I see after a crash:

Traceback (most recent call last):
  File "data_process/compile_results.py", line 577, in <module>
    wizard_data_compiler.generated_data_set(args.output_folder)
  File "data_process/compile_results.py", line 537, in generated_data_set
    compiled_data = self.compile_results()
  File "data_process/compile_results.py", line 492, in compile_results
    task_units_data.extend(self.get_units_data(task_unit))
  File "/private/home/komeili/dev/ParlAI/parlai/crowdsourcing/utils/analysis.py", line 191, in get_units_data
    unit_data = data_browser.get_data_from_unit(unit)
  File "/private/home/komeili/dev/Mephisto/mephisto/tools/data_browser.py", line 82, in get_data_from_unit
    ), f"Trying to get completed data from unassigned unit {unit}"
AssertionError: Trying to get completed data from unassigned unit MTurkUnit(247954, completed)

[JackUrb]

Is this data that you're loading from new runs? I don't imagine we should be running into many cases where a completed unit has no data anymore, but if not I'll have to revisit.

Otherwise, this looks fine to me!

[mojtaba-komeili]

Is this data that you're loading from new runs? I don't imagine we should be running into many cases where a completed unit has no data anymore, but if not I'll have to revisit.

Otherwise, this looks fine to me!

Yeah, it is using the new singleton system.