Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

First pass at integrating xP3 #60

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from
Draft

First pass at integrating xP3 #60

wants to merge 7 commits into from

Commits on Apr 14, 2023

  1. First pass at integrating xP3 

    - Add scripts to generate task constants for xP3 datasets
- Get started on a task_config generator for xP3:
  - integrates P3 config generator with existing hugging face loader
    @viraat
    viraat committed Apr 14, 2023
    Configuration menu
    Copy the full SHA
    100f8cd View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2023

  1. Add caching on the call to hf datasets for info

    @viraat
    viraat committed Apr 15, 2023
    Configuration menu
    Copy the full SHA
    17b471e View commit details
    Browse the repository at this point in the history

  2. Add progress bars

    @viraat
    viraat committed Apr 15, 2023
    Configuration menu
    Copy the full SHA
    e0b28a2 View commit details
    Browse the repository at this point in the history

  3. num_examples not present for certain datasets 

    - Add some try catch to log those datasets, and put dummy values
    @viraat
    viraat committed Apr 15, 2023
    Configuration menu
    Copy the full SHA
    8aed084 View commit details
    Browse the repository at this point in the history

  4. Temporarily take out GEM/xlsum as it doesn't have dataset_info

    @viraat
    viraat committed Apr 15, 2023
    Configuration menu
    Copy the full SHA
    ae113e4 View commit details
    Browse the repository at this point in the history

  5. Imagining final structure of constants to look like this

    @viraat
    viraat committed Apr 15, 2023
    Configuration menu
    Copy the full SHA
    ac984f5 View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2023

  1. Remove niv2 unzipped

    @viraat
    viraat committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    91d43e6 View commit details
    Browse the repository at this point in the history