Add support for presaved options (options aliases)

[stephenroller]

We already support for setting options from the command line.

Set up here:

ParlAI/parlai/core/params.py

Lines 648 to 654 in c2c800e

	parlai.add_argument(
	'-o',
	'--init-opt',
	default=None,
	help='Path to json file of options. '
	'Note: Further Command-line arguments override file-based options.',
	)

This is handled here:

ParlAI/parlai/core/params.py

Line 974 in c2c800e

def _load_known_opts(self, optfile, parsed):

We would like to be able to distribute options files with parlai.

• Make a new "options" folder inside parlai/
• Extend _load_known_opts so that when a user specifies "-o whatever" it looks inside parlai/options/whatever.opt to see if it can load options from there. Only do this if it is not a literal filename
• Extend _load_known_opts so that if there is a comma, it splits it and performs the load multiple times in sequence (-o presaved1,presaved2 should load presaved1.opt and then presaved2.opt
• Add a new generate_options.py inside docs, to generate a page which enumerates all the files in parlai/options and explicitly spells out what they are shorthand for. Put this as a new section inside "Scripts & CLI" called "Options shorthands" or similar

Additionally, please create a few initial options files for people to draw inspiration from:

• gen_blenderbot: shortcut for --beam-size 10 --inference beam --beam-block-ngram 3 --beam-context-block-ngram 3 --beam-min-length 20
• gen_meena: shortcut for --inference topk --beam-size 20 --topk 40

[klshuster]

Should this options/ folder have some sort of structure? For different purposes (e.g., generation/, init_options, task_options, etc.)

Also would be cool if there was an auto test for these options files that checked that the options were parsed correctly (so that when you add new options files you can be sure that the implementation didn't miss anything)

[stephenroller]

Let's just make sure we can support folders there, and let the structure evolve over time?

I'm really not sure what the right tradeoff of flat namespace vs proper hierarchy is. With a certain level of hierarchy, I would say the advantage of the shorthands are pretty much lost...

[klshuster]

i'd propose only file structure; on the command line, you would still be able to specify gen_blenderbot, which would search all subfolders

as i type this i realize that might obfuscate what's available. If we do not enforce file structure, then should we at least enforce options name structure? so all generation params must start gen_, all task options must start task_ etc?

[stephenroller]

I'd really rather have a 1:1 mapping between the folder and the utterance. so like gen/blenderbot if we have folders.

Let's set naming conventions in parley.

[github-actions]

This issue has not had activity in 30 days. Please feel free to reopen if you have more issues. You may apply the "never-stale" tag to prevent this from happening.

[klshuster]

This landed with #3564 so i think we can close