v1.1.1

Bug fix release

Fix a bug introduced in v1.1.0 caused by the removal of APEX, which increased the numerical instability of training with fp16. See #3532 for details.

v1.1.0

Breaking changes

• Removed dependence on APEX. --fp16-impl safe is the new standard (#3469)
• Change DialoGPT default to medium as that's the best results and most common setup (#3401)

Major new features

• Mutators are a new way to perform transformations on arbitrary datasets (#3448)
• Support for WandB (#3417)
• display_model now displays teacher-metrics, and world loggers store them (#3388, #3419)

Minor new features

• Make convenience script for dumping out chat to conversations format (#3354)
• Allow a variable number of transformer heads in distillation, and improved loss normalization (#3424, #3447, #3430)
• Better error messages when incorrectly providing a model zoo file (#3445)
• Add --no-backup option to vacuum. (#3431)
• Release BST human annotations (#3405)

Crowdsourcing changes

• Abstract class for compiling annotation results (#3422, #3423, #3464)
• Improvements to output format for some tasks (#3400, #3425)
• Configuration option for setting soft blocks (#3435)

Bug fixes

• [security] RCE fixed in chat services (#3402, #3429)
• Speedups when using conversation openers in self-chat (#3461)
• Speedups in metrics accumulation (#3474)
• Fix an error bounds error when manually applying vec2txt with --dict-tokenizer bytelevelbpe (#3437)
• Add pathmanager support to BERT agent (#3439)
• Fixes to Natural Questions (#3440, #3438)
• Remove a warning in --fp16-impl mem_efficient (#3443)
• Fix an issue with windows compatibility (#3428)
• Update data urls for Taskmaster-1 (-t taskmaster) (#3475)
• Fix a bug with terminal chat where the socket would close early (#3457)

Developer features

• New abstract method for ranking candidates in TorchGeneratorAgent (#3455)
• Improved type annotations and code cleanup (#3413, #3462, #3420)
• Various documentation immprovements (#3398, #3399, #3349, #3384, #3385, #3427, #3446, #3441)
• Test fixes (#3442, #3467)

v1.0.0

This marks a major release for ParlAI worthy of celebration. Thank you to the team, our numerous external contributors, and all of our users.

Major changes

MTurk is dead, long live Crowdsourcing

As the primary impetus for the release of v1.0.0, we have officially removed parlai.mturk from our repo, fully migrating to the newer, Mephisto-based parlai.crowdsourcing. If you have any questions about upgrading, please make sure to check the new tutorial and feel free to ask questions in our new Discussions forum

New discussions forum

We've enabled GitHub Discussions as a new place for individuals to ask more long-lived questions. We still encourage using issues for bug reports, but many "How do I...?" questions are better for Discussions now.

Distilled models

We have released several distilled BlenderBot models (#3313, #3314), as well as code for distilling your own models. See project page.

DECODE model releases

New project on addressing contradictions in dialogue models (#3324, #3325, #3327, #3334, #3309). See project page for more information.

Backwards incompatible changes

• parlai.mturk has been removed in favor of parlai.crowdsourcing (#3362, #3371)
• The API for add_cmdline_args has changed. Please see the PR for information on upgrading. This enables things like defaults that depend on other values, or arguments that depend on other arguments. (#3328)
• Improvements to world logger arguments (#3380)

New features in core

• New distinct metrics (including intra-distinct and inter-distinct). (#3347)
• 40% Speedups in --model-parallel true generations and training (#3326)
• Updated README, autodoc descriptions, PR templates (#3359, #3342, #3345, #3322))
• [developer] Many more task regression tests (#3240)

New features in crowdsourcing

• General improvements to ACUTE-Eval implementation, docs, and introduction of analysis framework (#3374, #3333, #3344, #3305, #3323)
• Conversion of turn annotations to Mephisto (#3262, #3356)
• Mephisto QA data collection (#3318, #3330)
• Mephisto image-chat task (#3320)
• Documentation improvements (#3319, #3373)
• Fixing flaky tests (#3363, #3372, #3321, #3355)

Smaller bugfixes

• Allow differing arguments for dynamic batching in eval/train (#3382)
• fix: deprecated argument when calling transformers>4.0.0 (#3393)
• Small bugs with crowdsourcing (#3346, #3386, #3387)
• Fix a misparse of -o (#3347)
• Fix bert models in distributed. (#3351)
• Suppress PermissionErrors in windows deletions. (#3340)
• Fix for Interactive Web with TF-IDF models (#3381)
• Fix a small bug in Wizard of Wikipedia task (#3100)
• Fix minor idx not-existing issue in world_logging (#3317)

v0.10.0

Backwards compatibility breakages:

• TorchClassifierAgent changed use_reply default from 'label' to 'none' (#3128)
• [BERT] Deprecate Adam with Decay (#3235)

Major features

• [safety] Open source Sensitive Topics classifier and data (#3253, #3259, #3260, #3258, #3230, #3227, #3252)
• Support for model distillation (#3274)
• [docs] Link to request Multi-Modal BlenderBot weights (#3281)

Minor features

• DialogTeacher provides verbose warning if opt['datafile'] is not set (#3140)
• [bpe] Support BPE dropout (#3232)
• Support special tokens in non-HF BPE dictionaries. (#3213)
• Unify verbose & display_verbose, enable verbose during interactive (#3205, #3248)
• BART Supports incremental decoding (#3231)
• Use more modern metrics reporting with data_stats. (#2854)
• Ease-of-use improvements to wrapper teacher (#3247)
• Add classifier for style-controlled generation (#2829)
• Add support for frozen encoders (only train head) in TorchClassifierAgent (#3215)

Mephisto Migrations & Upgrades

• [CrowdSourcing] Turn annotation edits (#3234)
• Fast ACUTE in Mephisto (#3297)
• Mephisto API upgrades (#3275, #3285)
• Remove two old MTurk tasks (#3225)
• Port static turn annotations to Hydra (#3149)
• Add command for viewing all Hydra parameters (#3210)
• Analysis script for static turn annotations (#3263)
• Make the log rate an adjustable parameter (#3189)

Bug Fixes:

• Fix argparse issues in python 3.8 (#3284)
• Torch 1.8 Compatibility (#3228)
• Fix dialogpt dual usage of END_IDX (#3256), broken by adding support by adding batchsize > 1
• Fix minor bugs with GPT2 (#3261, #3207, #3208)
• Fix an issue where ChunkTeacher would sometimes repeat examples (#3244, #3301)
• make dir at start of dict.save (#3250)
• Fix a bug in distributed sync when metrics dicts become large (#3291)
• [BART] Some smaller bug fixes (#3272, #3236)
• Data parallel is now false in Offensive Language Classifier (#3212)
• Fix a bug with LR scheduler loading from disk in TCA (#3229)

Developer changes:

• Task regression testing (#3223)
• Stabilized distributed gpt2 test (#3219)
• CI for Mephisto tasks (#3206, #3238, #3246, #3254)
• CircleCI refactor (#3245)
• Documentation improvements (#3226, #3251)
• Deprecated some dead code (#3257)
• Various CI fixes (#3310, #3264)

v0.9.4

v0.9.4 release

Known issues

• Short options like -m and -t do fail in Python 3.8. Use --model and --task

New project/paper releases

• Safety recipes project (#3190, #3199)
• Gender bias project (#3171, #3146, #3131))
• Multimodal Blender Bot (#3152, #3151, #3144)
• LIGHT Quests project (#3153, #3156)

Major MTurk changes

• [Removal] Deleting 2/3 of MTurk (#3161)
• [Upgrade] Porting ACUTE-Eval task to Hydra (#3103)
• [Task] Adding option for file with annotation indices per conversation and other features to turn annotations static (#3154, #3142)
• [Docs] AcceptabilityChecker doc (#3124)

New Features

• [ParlAI] Adding tensorboard_logdir argument to TensorboardLogger. (#3184)
• [ParlAI] [TGA] Return text for all beams (#3123, #3179)
• [ParlAI] Using the parlai color (#3137)
• [ParlAI] Add a --version flag to the parlai command #3163 (#3164)
• [Task] Improvements to Taskmaster-2 (#3135, #3134, #3129)
• [Task] Flag to enable polarity categories in ImageChat (#3132)
• [Model] Allow Polyencoder to resize token embs (#3194)

Bug Fixes

• [Model] Fix GPT-2 generation when using batchsize >1
• [ParlAI] Show subcommand help error messages on parse error in supercommand. (#3196)
• [ParlAI] Fixes to --init-model options and --init-opt. (#3162, #3112)
• [ParlAI] Update requests version (#3155)
• [HuggingFace/GPT2] Check directory depending on size param for gpt2 files (#3158)
• [Docs] Correct swapped README links (#3166)
• [Task] ConvAI2: Bug fix in convai2:normalized NormalizedTeacher (#3168)
• [Task] COCO: Make sure folder is always created when unzipping COCO (#3143)

Developer notes & Tests

• [Tests] New/fixed tests (#3167, #3117, #3114, #3082)
• [Feature] Add ability for Metrics to add metrics from another Metrics object (#3145)
• [Docs] Comments in AbstractWrapperTeacher.act() (#3160)
• [Docs] Documentation improvements to poly-encoder (#3159)
• [Quality] distributed_eval now mimics distributed_train (#3157)
• [Quality] Add "fb" option to autoformat script for parlai_fb (#3165)

v0.9.3

Known issues

• Short options like -m and -t do fail in Python 3.8. Use --model and --task

Breaking Changes

• A number of old MTurk tasks have been archived and removed from the code (#3085)

New Features

• [image] Detectron feature extraction (#3083)
• [data] Natural questions (#3070)
• [data] TaskMaster-2 (#2678)
• [data] New versions of multiwoz (#3072)
• [distributed] Allow non-tcp based distributed setup (#3095)
• [core] Move torch.load/torch.save to PathManager. (#3094, #3077)
• [mturk] New task on static turn annotations (#3053)
• [mturk] New features in human+model annotation (#3006)
• [core] TorchClassifierAgent now prints its number of prameters (#3086)

Doc Changes:

• New Worlds tutorial (#3049)
• Tutorial on using -t jsonfile (#3061)
• Better help message for --init-model (#3090)
• Additions to FAQ (#3073)
• Updated model zoo descriptions for BlenderBot (#3096)

Bug Fixes

• Distributed evaluation now writes to world logs earlier (#3122)
• An argument was updated from store_true to bool (#3113)
• Self-chat now fails loudly with unexpected batchsize (#3081)
• Update drqa default tokenizer away from removed (#3069)
• Using wizard of wikipedia in interactive mode downloads data (#3079)

Developer notes:

• New pre-commit git-secrets (#3106)
• Code coverage improvements (#3110, #3091)
• More reliable tests. (#3108, #3097, #3055)
• Mephisto task dependencies have been updates due to security bugs (#3111, #3101, #3104)
• MTurk config folders are exempt from init.py requirements (#3105)

v0.9.2

v0.9.2 release notes

Breaking changes:

• Fix no-op mask fill in some of the the transformer/ranker models (#3068). We believe this to be fine, but it may change the performance of some ranking models. If you are training a transformer/ranker or polyencoder model and you notice your model no longer gets the same performance, please file an issue on GitHub.

Bugfixes:

• Fix a bug where using tensorboard in training caused a crash (#3067)
• Fix a bug where BART failed to load from checkpoint properly (#3060)
• Fix an issue where some arguments could not be overridden in BART (#3065)

Other:

• fromfile now emits a warning about using valid/test (#3063)

v0.9.1

Version 0.9.1 Changelog:

ParlAI now supports pytorch 1.6.

Breaking changes:

• Change the arguments for JsonFile teacher, and fix some bugs (#3054)
• ParlAI now expects HuggingFace tokenizers >=0.8.0 (#2967)

New features/Improvements:

• Release of LIGHT-WILD project (#2992)
• HRED has now been added as a standard model (#2989)
• DialoGPT has been added as a standard model (#3007)
• Allow colored logging in jupyter notebooks. (#3024)
• [Squad2] Add option to change label for impossible answers (#3020)
• ChunkTeacher has support to prevent auto enqueuing (#2993)
• Add support for PathManager. (#3011)
• DistilGPT2 has been added as an option of GPT2 model (#2997)
• tfidf_retriever now reports doc ids (#3008)

Lots of new/updated documentation

• Custom metrics tutorial (#2975)
• Chat Services tutorial clarification (#3048)
• Release of Mephisto ACUTE-Evals (#3002)
• Writing custom scripts and integration with parlai command (#3046)
• Writing tests for ParlAI (#3044)
• Contributor guide (#3047)
• Mark deprecation of ParlAI MTurk for Mephisto (#2988)
• Updates to docs on how to build docs (#3041)
• Update Link and Commands Display format (#2952)
• Small modifications (#3036, #2994, #3031, #2916)

Bugfixes:

• Upgrade to pytorch 1.6 (#3022)
• Linear/Cosine LR schedulers don't crash in distributed (#3000)
• Fix unexpected crashes in convo_render (#3052)
• Context block segfault fix (#3019)
• Fix crash in loading from checkpoint in Cosine/Linear LR schedulers (#3025)
• Upgrade schedulers to be Pytorch 1.6 friendly (#3025)
• Fix bug where "restart conversation" didn't work in chat_services browser (#3050)
• Faster amazon QA teacher (#2851)
• Fix crashes to Reward Unlikelihood agent (#3034)
• Fix a misalignment in BART agent between generation and training (#3021)
• Prevent mixing model parallel and multiprocessing. (#2964)
• [ACUTE][Mephisto] Fix extend that should append (#3029)
• Adding option to load pretrained model from directory for gpt2 (#3027)
• Fix an obscure crash with dict_file and GPT2 (#3038)

Developer:

• Improve coverage of some base agents (#3026)
• CircleCI now produces a build of website as an artifact (#3035)
• Rework sphinx docs and upgrade docs to MarkDown (#2972, #2999, #3003)
• Minor refactor of transformer decoder forward (#3009)

v0.9.0

v0.9.0 ChangeLog

Breaking Changes

• Removed support for HogWild and ExecutableWorld (#2919)
• Archived a number of older projects (Mastering the dungeon, TalkTheWalk, Memnn_feed) (#2919)

New Features

• [Dynamic Batching] Add world logging capability (#2941)
• [Torch Generator Agent] Option to block full context (#2928)
• [Safety] can supply custom offensive language file (#2968)
• [Safety] Add new model file argument to safety classifiers (#2955)
• Script for comparing two different opts (#2932)
• [Image S2S] Standardize Image Seq2Seq Image Processing (#2911)

Bug fixes

• [Chunk teacher] Bug with distributed evaluation (#2935)
• Add encoding to "with open"s in teachers.py (#2946)
• Fix an issue where parlai suppercommand wasn't handling set_params appropriately (#2956)
• Fix a bug where Git was trying to be called regard (#2961)
• Project specific: Personality Captions interactive mode fix (#2963)
• Project specific: fix openers loading opt (#2965)
• Fix underscore opt not being handled by parlai supercommand (#2938)
• [Style] Add style-controlled generation agent (#2913)
• Add return in report of wrapper agents (#2907)

Doc updates

• Fixed a mistake in documentation for creating tasks based on ParlAIDialogTeacher. (#2973)
• Updated link for LICENSE file (#2970)
• Added pip installation guide to Readme.md (#2959)
• Mention colab tutorial in our quickstart tutorial (#2936)

Other

• Tweak colors of logs. (#2940)
• Add printing of opts into scripts. (#2933)
• [Dodeca] Split Dodeca tar file (#2934)
• [BART/TGA] BART and TGA API Updates (#2840)
• Show all stdout on GPU tests (#2929)

v0.8.0

Tying off a "pieces of 8" release, with an arbitrary version number to switch to semantic versioning.

Notable additions:

• Fix a bug in distributed launching (#2850)
• Add support for reversed history in TorchAgent (--history-reversed true), which reverses text_vec. (#2786, #2888 )
• A new parlai super command, with a script registry. This replaces all the usage of "examples/train_model.py" etc. (#2859, #2845, #2860, #2887, #2889)
• Some improvements to model parallelization (#2839)
• Add some timing metrics, including tps (tokens per second), etc. (#2878)
• Self-chat can set up two models talking to each other with different options (#2803)
• An mturk task with per-turn annotations (#2877, #2886)
• An abstract class for writing "wrapper" teachers, which contain some transformation of data (#2842)
• Support writing to a log in an interactive script (#2881)

Bug fixes:

• Fix a type error in valid_step (#2890)
• Improve the output when listing differences between an init model and specified options (#2880)
• Upgrade TGA to be compatible with pytorch 1.6 (#2902)
• Fix a bug in seq2seq that broke with pytorch 1.6 (#2910)

Breaking changes:

• Switch --load-from-checkpoint to True by default in train_model
• Upgrade dependency to pillow 7.2.0 for a security reason (#2896)
• Switch to using a suffix on the model file, instead of a NamedTemporaryFile, for atomic save (#2898)
• Removed some old deprecated pytorch data teacher text (#2883)