Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

[WoW] WoW filter for examples with *only* no passages used #4418

Merged
merged 1 commit into from
Mar 14, 2022

Conversation

klshuster
Copy link
Contributor

Patch description
Add mutator for only including examples in which the wizard does not use knowledge.

Testing steps
Tested mutator locally, confirmed desired effects.

$ parlai dd -t wizard_of_wikipedia --mutators wow_only_no_passage_used -v
- - - - - - - END OF EPISODE - - - - - - - - - -
~~
[id]: wizard_of_wikipedia
[checked_sentence]: no_passages_used
[chosen_topic]: Pizza
[knowledge]: Pizza Pizza is a traditional Italian dish consisting of a yeasted flatbread typically topped with tomato sauce and cheese and baked in an oven.
Pizza It can also be topped with additional vegetables, meats, and condiments, and can be made without cheese.
Pizza The term "pizza" was first recorded in the 10th century, in a Latin manuscript from the Southern Italy town of Gaeta in Lazio, on the border with Campania.
Pizza Modern pizza was invented in Naples, and the dish and its variants have since become popular and common in many areas of the world.
Pizza In 2009, upon Italy's request, Neapolitan pizza was registered with the European Union as a Traditional Speciality Guaranteed dish.
Pizza "Associazione Verace Pizza Napoletana" (True Neapolitan Pizza Association), a non-profit ...
pizza.
Pizza Restaurants or chains specializing in pizza are pizzerias.
Pizza Pizza delivery is common in some parts of the world.
Pizza Pizza is sold fresh or frozen, either whole or in portions.

[title]: no_passages_used
[text]: Pizza
[labels]: anything with sauce and cheese on it is a pizza. Anything
- - - - - - - END OF EPISODE - - - - - - - - - -
~~
[id]: wizard_of_wikipedia
[checked_sentence]: no_passages_used
[chosen_topic]: Hitchhiking
[knowledge]: Hitchhiking Hitchhiking (also known as thumbing, hitching, or autostop) is a means of transportation that is gained by asking people, usually strangers, for a ride in their automobile or other vehicle.
Hitchhiking A ride is usually, but not always, free.
Hitchhiking Itinerants have also used hitchhiking as a primary mode of travel for the better part of the last century, and continue to do so today.
Hitchhiking The hitchhikers' methods of signaling to drivers differ around the world.
Hitchhiking Many hitchhikers use various hand signals.
Hitchhiking If the hitchhiker wishes to indicate that he needs a ride, he may simply make a physical gesture or display a written sign.
Hitchhiking In North America, United Kingdom and most of Europe, the gesture involves extending the arm toward the road ...
and Roy Norris Norris accepted a plea bargain whereby he agreed to testify against Bittaker and was sentenced to life imprisonment on May 7, 1980, with possibility of parole after serving 30 years.

[title]: no_passages_used
[text]: I do not like hitchhiker, I don't mind if they try to hitchhike, but I won't be the person transporting them because there is a potential chance of danger for little to no reward.
[labels]: I agree with you; I also feel that hitchhiking can be very dangerous for the hitchhikers - like the three Israeli teenagers in 2014
- - - - - - - END OF EPISODE - - - - - - - - - -
~~
[id]: wizard_of_wikipedia
[checked_sentence]: no_passages_used
[chosen_topic]: Bible
[knowledge]: Bible The Bible (from Koine Greek τὰ βιβλία, "tà biblía", "the books") is a collection of sacred texts or scriptures that Jews and Christians consider to be a product of divine inspiration and a record of the relationship between God and humans.
Bible Many different authors contributed to the Bible.
Bible What is regarded as canonical text differs depending on traditions and groups; a number of Bible canons have evolved, with overlapping and diverging contents.
Bible The Christian Old Testament overlaps with the Hebrew Bible and the Greek Septuagint; the Hebrew Bible is known in Judaism as the Tanakh.
Bible The New Testament is a collection of writings by early Christians, believed to be mostly Jewish disciples of Christ, written in first-century Koine Greek.
Bible These early Christian ...
can be considered inerrant, because, as the statement reads: "the autographic text of Scripture, ... in the providence of God can be ascertained from available manuscripts with great accuracy".

[title]: no_passages_used
[text]: Is it true that it took quite alot of work to make it as widely available as it is today? Which country made that possible?
[labels]: There is no country in particular, but they all contributed to make the Bible one of if not the best selling book of all time!
- - - - - - - END OF EPISODE - - - - - - - - - -

etc...

Copy link
Contributor

@mojtaba-komeili mojtaba-komeili left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
But why did you call that function fails? The code is clear, but would you mind renaming it to something like no_knowledge_used?

@klshuster
Copy link
Contributor Author

I used the same base mutator for both "only knowledge used" and "no knowledge used", and they each define their own failure function, fails

@klshuster klshuster merged commit 52c9bcd into main Mar 14, 2022
@klshuster klshuster deleted the wiz_wiki_mutator branch March 14, 2022 17:03
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants