Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

Patch on world logging #3674

Merged
merged 2 commits into from
Jun 4, 2021
Merged

Patch on world logging #3674

merged 2 commits into from
Jun 4, 2021

Conversation

jxmsML
Copy link
Contributor

@jxmsML jxmsML commented May 26, 2021

Patch on world logging

  • filter by message.is_padding() when writing episodes to self._current_episodes.
  • modify the test tests/test_eval_model.py on test_save_report to verify the no dummy world logs are stored (those with batch_padding True)

before this patch:
the world logs saved will contain the following dummy episode:

{"dialog": [[{"batch_padding": true, "episode_done": true, "id": "bot_adversarial_dialogue:HumanSafetyEvaluation.persona_False_flatten_False"}, {"id": "TransformerGenerator", "episode_done": false}]], "context": [], "metadata_path": "random/world_logs.metadata"}

After this patch, no such padding examples will be saved to the world logs

Testing steps

Other information

@@ -74,12 +75,17 @@ def _add_msgs(self, acts, idx=0):
"""
msgs = []
for act in acts:
# padding examples in the episode[0]
if isinstance(act, Message) and act.is_padding():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to cast it as a message in case it happens to be a dict

@jxmsML jxmsML merged commit a162ee6 into master Jun 4, 2021
@jxmsML jxmsML deleted the convo_log_pad branch June 4, 2021 00:29
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants