v0.2.5

Changed

• BERTScore now loads the model weights in the construction instead of each time the scoring method is called.

v0.2.4

Added

• Added saving the Fabbri data with the original reference summaries and documents
• Adding some annotations from MultiLing 2019
• Added Dockerized versions of ROUGE and QAEval
• Added annotations from Kryscinski et al. (2019).
• Added a Dockerized version of BERTScore
• Added a Dockerized version of BLEURT
• Added a Dockerized version of MoverScore
• Added a Dockerized version of BARTScore
• Added a Dockerized version of Lite3Pyramid

Changed

• Moved the actual QAEval metric implementation into the qaeval library.
  The new minimum version is qaeval==0.1.0.

Removed

• Removed DecomposedRouge, which can now be found here.

v0.2.3

Added

• Added SentBLEU under the name sent-bleu.
• Added saving all of the summaries from the 16 models in Fabbri et al. (2020).
• Added saving all of the summaries from the data in Bhandari et al. (2020).

Changed

• Updated Blanc to use blanc==0.2.1.
• Setting up fabbri2020 now automatically downloads the tar files

Removed

• Removed idf support for BERTScore so we can remove the dependence on our fork of the original repo.

v0.2.2

Added

• Added a verbose option to QAEval

Changed

• Changed QAEval to use the updated qaeval interface with version 0.0.8.
  The QA results will now include the answer offsets.

v0.2.1

Added

• Added the New York Times dataset. See here.
• Added better tutorials for using the library.

Changed

• Added an exception with an error message if PyrEval is used with a single reference summary.

Fixed

• Fix the overrides package to version 3.1.0 to fix a bug that was caused in the Params class in overrides version 6.0.0

v0.2.0

Added

• Added the annotations collected by Bhandari et al., (2020).
• Added BLANC
• Added the annotations collected by the BLANC paper.
• Added a wrapper around the implementation of APES.
• Added the Multi-News dataset.
• Added the WCEP dataset
• Added confidence interval calculation and running hypothesis tests for the correlation coefficients

Changed

• Changed the backend for the correlation calculation to use matrices instead of the MetricsDicts

Fixed

• Fixed a bug in which QAEval would crash if you don't use LERC

v0.1.5

Fixed

• Including the LERC output from the individual QA pairs in QAEval

v0.1.4

Added

• Added scoring QAEval predictions with LERC

Fixed

• Creating the .sacrerouge/metrics directory in the BLEURT setup script if it doesn't exist.

v0.1.3

Bumping version

Adding DecomposedRouge

Adding the DecomposedRouge metric from https://arxiv.org/abs/2010.12495