Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added WMT News Crawl Dataset for language modeling #688

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open

Added WMT News Crawl Dataset for language modeling #688

wants to merge 22 commits into from

Commits on Feb 3, 2020

  1. Added WMT News Crawl Dataset for language modeling

    @anmolsjoshi
    anmolsjoshi committed Feb 3, 2020
    Configuration menu
    Copy the full SHA
    4f5a3d3 View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2020

  1. Removed WMT from torchtext.datasets

    @anmolsjoshi
    anmolsjoshi committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    40a0b3e View commit details
    Browse the repository at this point in the history

  2. Fixed error related to tar files.

    @anmolsjoshi
    anmolsjoshi committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    b498950 View commit details
    Browse the repository at this point in the history

  3. Added WMT to experimental dataset

    @anmolsjoshi
    anmolsjoshi committed Feb 4, 2020
    Configuration menu
    Copy the full SHA
    1a016d5 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2020

  1. Working tests and text classification dataset

    @anmolsjoshi
    anmolsjoshi committed Feb 5, 2020
    Configuration menu
    Copy the full SHA
    e113246 View commit details
    Browse the repository at this point in the history

  2. Updated docstrings

    @anmolsjoshi
    anmolsjoshi committed Feb 5, 2020
    Configuration menu
    Copy the full SHA
    71eeaf3 View commit details
    Browse the repository at this point in the history

  3. Added line to join root with extracted files

    @anmolsjoshi
    anmolsjoshi committed Feb 5, 2020
    Configuration menu
    Copy the full SHA
    4c807a9 View commit details
    Browse the repository at this point in the history

Commits on Feb 24, 2020

  1. Merge branch 'master' of https://github.com/pytorch/text into feature… 

    …/news_crawl
    @anmolsjoshi
    anmolsjoshi committed Feb 24, 2020
    Configuration menu
    Copy the full SHA
    8b2e372 View commit details
    Browse the repository at this point in the history

  2. Incorporated comments

    @anmolsjoshi
    anmolsjoshi committed Feb 24, 2020
    Configuration menu
    Copy the full SHA
    960ab99 View commit details
    Browse the repository at this point in the history

  3. Reverted files to master version

    @anmolsjoshi
    anmolsjoshi committed Feb 24, 2020
    Configuration menu
    Copy the full SHA
    3b39db2 View commit details
    Browse the repository at this point in the history

  4. Revered test files to master version

    @anmolsjoshi
    anmolsjoshi committed Feb 24, 2020
    Configuration menu
    Copy the full SHA
    061fff1 View commit details
    Browse the repository at this point in the history

  5. spacing

    @anmolsjoshi
    anmolsjoshi committed Feb 24, 2020
    Configuration menu
    Copy the full SHA
    04f497c View commit details
    Browse the repository at this point in the history

  6. Fixed arguments

    @anmolsjoshi
    anmolsjoshi committed Feb 24, 2020
    Configuration menu
    Copy the full SHA
    d259c7d View commit details
    Browse the repository at this point in the history

  7. fixed flake8 errors

    @anmolsjoshi
    anmolsjoshi committed Feb 24, 2020
    Configuration menu
    Copy the full SHA
    a7d720c View commit details
    Browse the repository at this point in the history

Commits on Feb 25, 2020

  1. Added test for WMTNewsCrawl

    @anmolsjoshi
    anmolsjoshi committed Feb 25, 2020
    Configuration menu
    Copy the full SHA
    e6308d1 View commit details
    Browse the repository at this point in the history

  2. fixed flake8 issues

    @anmolsjoshi
    anmolsjoshi committed Feb 25, 2020
    Configuration menu
    Copy the full SHA
    27fb2d3 View commit details
    Browse the repository at this point in the history

  3. Added a test for incorrect option for data_select

    @anmolsjoshi
    anmolsjoshi committed Feb 25, 2020
    Configuration menu
    Copy the full SHA
    5be9f51 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into feature/news_crawl

    @anmolsjoshi
    anmolsjoshi authored Feb 25, 2020
    Configuration menu
    Copy the full SHA
    62b20e9 View commit details
    Browse the repository at this point in the history

Commits on Feb 26, 2020

  1. Added option for year for WMT, included information by year

    @anmolsjoshi
    anmolsjoshi committed Feb 26, 2020
    Configuration menu
    Copy the full SHA
    de521be View commit details
    Browse the repository at this point in the history

  2. Updated Example code

    @anmolsjoshi
    anmolsjoshi committed Feb 26, 2020
    Configuration menu
    Copy the full SHA
    c228e80 View commit details
    Browse the repository at this point in the history

  3. Added validation for language and year with tests

    @anmolsjoshi
    anmolsjoshi committed Feb 26, 2020
    Configuration menu
    Copy the full SHA
    b5e8a02 View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2020

  1. Merge branch 'master' into feature/news_crawl

    @anmolsjoshi
    anmolsjoshi authored Apr 15, 2020
    Configuration menu
    Copy the full SHA
    2a2cbea View commit details
    Browse the repository at this point in the history