Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

Add gpu option to torchscript BART models #3979

Merged
merged 11 commits into from
Aug 26, 2021
Merged

Add gpu option to torchscript BART models #3979

merged 11 commits into from
Aug 26, 2021

Conversation

maryamdaneshi
Copy link
Contributor

@maryamdaneshi maryamdaneshi commented Aug 25, 2021

Patch description
Parsing no_cuda to enable/disable scripting for gpu. additionally, make the changes needed to script for gpu

Testing steps

export gpu model

parlai torchscript \
--model-file ${MODEL_FILE} \
--model bart \
--no-cuda false \
--scripted-model-file ~/_test_scripted_model__bart.pt \
--input 'I am looking for a restaurant in the west part of town.|APIRESP: Restaurant 14 matches'

Interact with the model

parlai interactive \
--model-file ${SCRIPTED_MODEL_FILE} \
--model bart \
--no-cuda false

output

Generating given the original unscripted module and the scripted module is the same, unit tests to prove that 

Copy link
Contributor

@EricMichaelSmith EricMichaelSmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thanks! Minor comment

}
if opt.get("script_for_gpu", False):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm instead of adding the --script-for-gpu flag, could we just reuse the value of the no_cuda key?

@stephenroller
Copy link
Contributor

We should add a GPU test

.circleci/config.yml Outdated Show resolved Hide resolved
@maryamdaneshi maryamdaneshi merged commit 19d7055 into master Aug 26, 2021
@maryamdaneshi maryamdaneshi deleted the gpu_script branch August 26, 2021 01:04
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants