Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix mirrored pair input #1031

Merged
merged 1 commit into from
Mar 9, 2020
Merged

Conversation

HaokunLiu
Copy link
Member

When discussion with Phil about the preprocess function, I found I wrote a typo when implementing the preprocessing of gpt2-like single directional model.

This will have slight negatively affect on the performance of gpt, gpt2 and transformer-xl on sts-b, mrpc, qqp. But not other transformer models or other tasks.

Copy link
Contributor

@pyeres pyeres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed that this change reorders inputs in line with what's described in GPT (section 3.3).

@pyeres pyeres merged commit 57ea962 into master Mar 9, 2020
@pyeres pyeres deleted the fix_mirrored_pair_option_in_preprocess branch March 9, 2020 22:59
phu-pmh pushed a commit that referenced this pull request Apr 17, 2020
@jeswan jeswan added the jiant-v1-legacy Relevant to versions <= v1.3.2 label Sep 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
jiant-v1-legacy Relevant to versions <= v1.3.2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants