Custom retrieval corpus support, FActScore is now published at EMNLP!

This update fixes a few bugs in our custom retrieval corpus support, and adds the ACL anthology as an example (see #24).

In other news, FActScore was recently accepted as a main conference paper in EMNLP 2023! This release additionally updates our PyPI webpage to reflect this.

Length penalty in FActScore, abstain detection customization, bug fixes

This update adds a few new important features to our PIP package (thanks again to our beta users!):

• (#21) To penalize very short generations, we have added a length penalty similar to the BLEU scores. This is enabled by default and can be customized using the --gamma flag. See here for more details.
• (#20) Users can now customize their abstain detection workflow using the --abstain_detection flag. We have added two abstain detection functions (generic, perplexity_ai), but disabled them by default. See here for more details.
• (d768d81) bug fixes in custom corpus retrieval

Instruct LLAMA integration, more informative logging (metrics, API cost estimation, customization)

This update to our PIP package contains a number of new features requested by our beta users:

• You can now use Instruct LLAMA as well besides ChatGPT! Set --model_name retrieval+llama+npm to use it. The factscore.download_data script has been fully updated to also install Instruct LLAMA if --llama_7B_HF_path is set as shown in the README.

• We have add a lot more information into our logging, such as alternative metrics (respond ratio, # of generated facts) and OpenAI API cost estimation. We have also moved to the logging package and have a few flags to ensure a nicer logging experience (--print_rate_limit_error and --verbose).

• More customization of cache is now possible, with --model_dir and --data_dir directories.