Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

Generate model card #3884

Merged
merged 19 commits into from
Aug 19, 2021
Merged

Generate model card #3884

merged 19 commits into from
Aug 19, 2021

Conversation

liliarose
Copy link
Contributor

@liliarose liliarose commented Jul 30, 2021

Patch description

Moving from internal repo to external. + a couple bug fixes

Also added comments to places that r new (I think I caught most of them, but didn't mark everything that was moved inside/outside the function) + also added a lot of comments :)

Tests

  1. Generation report mode with single tasks
  2. generation report mode with 2 tests (just test once to make sure it works; not doing safety_bench b/c it takes too long
  3. Generation of model card

Logs

  1. parlai gmc -mf zoo:blender/blender_90M/model -fts sample_model_cards/blender_90M --mode gen:data_stats -ne 1000

Screen Shot 2021-07-30 at 2 49 21 PM

Screen Shot 2021-07-30 at 2 49 45 PM

  1. parlai gmc -mf zoo:blender/blender_90M/model -fts sample_model_cards/blender_90M --mode gen:eval -bs 128 -ne 256

Screen Shot 2021-07-30 at 3 11 21 PM

Screen Shot 2021-07-30 at 3 13 28 PM

  1. parlai gmc -mf zoo:blender/blender_90M/model -fts sample_model_cards/blender_90M --mode gen:safety_bench --debug True --wrapper blenderbot_90M
    (copied the same args.json)
    NOTE THAT THIS WAS DONE IN DEBUGGING MODE FOR FASTER TESTING

Screen Shot 2021-07-30 at 3 18 09 PM

Screen Shot 2021-07-30 at 3 18 56 PM

  1. parlai gmc -mf zoo:blender/blender_90M/model -fts sample_model_cards/blender_90M --mode gen:sample

Screen Shot 2021-08-03 at 10 54 23 PM

Screen Shot 2021-08-03 at 9 38 32 PM

  1. parlai gmc -mf zoo:blender/blender_90M/model -fts sample_model_cards/blender_90M --mode gen:sample:safety_bench -ne 1000 --debug True --wrapper blenderbot_90M

Screen Shot 2021-08-04 at 8 47 37 AM

  1. Model Card: parlai gmc -mf zoo:blender/blender_90M/model -fts sample_model_cards/blender_90M

https://gist.github.com/liliarose/f9daeb5cc21a7dc438dbf23f3976797c (too lazy to add the image + safety_benchmark was in debug mode, so the numbers weren't correct)

@liliarose liliarose force-pushed the generate_model_card branch 3 times, most recently from f196f19 to 6cec9f2 Compare August 2, 2021 23:52
@liliarose liliarose marked this pull request as ready for review August 3, 2021 17:36
@liliarose liliarose requested review from jxmsML, EricMichaelSmith and meganung and removed request for EricMichaelSmith, meganung and jxmsML August 3, 2021 17:44
@liliarose liliarose marked this pull request as draft August 3, 2021 17:46
@liliarose liliarose self-assigned this Aug 3, 2021
@liliarose liliarose force-pushed the generate_model_card branch 3 times, most recently from 542d330 to 6c30d74 Compare August 4, 2021 05:02
@liliarose liliarose force-pushed the generate_model_card branch from 6c30d74 to 27cef31 Compare August 4, 2021 05:03
@liliarose liliarose requested a review from jxmsML August 4, 2021 23:25
if defaults.get(section):
return create_warning(defaults.get(section))
if self.mode != M_final:
msg = f'Missing {section}: Probably need to be grabbed from paper & added by u (the creator)'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit- u -> you

@facebook-github-bot
Copy link

Hi @liliarose!

Thank you for your pull request.

We require contributors to sign our Contributor License Agreement, and yours needs attention.

You currently have a record in our system, but the CLA is no longer valid, and will need to be resubmitted.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

Copy link
Contributor

@jxmsML jxmsML left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@liliarose 🚀

@jxmsML jxmsML merged commit 0c0ab5c into master Aug 19, 2021
@jxmsML jxmsML deleted the generate_model_card branch August 19, 2021 21:16
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants