Skip to content

Commit

Permalink
enhances and documents recipes (#815)
Browse files Browse the repository at this point in the history
In case if you happen to know already what recipes in bap are, then
the main chainges are summarized below:

- the recipe is no longer requred to be a zip file, plain files, or
  directories are now accepted

- the recipes interface is now documented and a couple of issues were
  fixed during the process

- filenames are no longer substituted, however there is now a builtin
  parameter `$prefix` that is substituted with the path to the recipe
  root folder.

For the rest, I'm copypasting excerpts from the bap man page, that
describe the recipe system in detail as well as few motivational
examples

```
RECIPE DESCRIPTION
       A recipe is either a single file or a directory (optionally zipped)
       that contains a parametrized specification of command line parameters
       and support files if necessary.

       The purpose of the recipe is to make bap runs reproducible, so that you
       can share one simple file - the recipe, instead of a bunch of scripts
       together with some verbal instructions. Since recipes introduce an
       extra layer of abstraction they actually simplify interaction with bap
       by hiding unnecessary details. Recipes also provide a mechanism for
       building ready solutions and sharing them with users.

       To use a recipe, just specify its name using the --recipe command line
       parameter. If a recipe has parameters then they could be specified as
       colon separated list of <key>=<value> pairs. See the --recipe parameter
       for more information. To read the recipe description, including the
       list of parameters and resulting command line, use the --show-recipe
       command line option. To list all installed recipes use the
       --list-recipes option.

       The main (and the only necessary) part of a recipe is the recipe
       specification, that is a file that contains a list of recipe items in
       an arbitrary order. Each item is either a command line option, a
       parameter, or a reference to another recipe. All items share the same
       syntax - they are flat s-expressions, i.e., a whitespace separated list
       of strings enclosed in parentheses. The first string in the lists
       denotes the type of the item. E.g., (option run-entry-points malloc
       calloc free).

       The option command requires one mandatory parameter, the option name,
       and an arbitrary number of arguments that will be passed to the
       corresponding command line option. If there are more than one argument
       then they will be concatenated with the comman symbol, e.g., (option
       opt a b c d) will be translated to --opt=a,b,c,d. Option arguments may
       contain substitution symbols. A subsitution symbol starts with the
       dollar sign, that is followed by a named (optionally delimited with
       curly braces, to disambiguate it from the rest of the argument). There
       is one built in parameter called prefix, that is substituted with the
       path to the recipe top folder. See the parameter command to learn how
       to introduce parameters.

       The parameter command introduces a parameter to the recipe, i.e., a
       variable ingredient that could be changed when the recipe is used. The
       parameter command has 3 arguments, all required. The first argument is
       the parameter name, the second is the default value, that is used if
       the a parameter wasn't set, and the last argument is the parameter
       description. The substitution symbol will be replaced with the default
       value of a parameter, if a value of the parameter wasn't passed through
       the command line. Example,

           (parameter depth 128 "maximum depth of analysis")
           (option analysis-depth depth)

       If the parameter is not set through the command line, then it will be
       substituted with 128 otherwise it will receive whatever value a user
       has passed. Finally, the extend command is like the include statement
       in the C preprocessor as it includes all the ingredients from another
       recipe. (Make sure that you're not introducing loops!). The command has
       one mandatory argument, the name of the recipe to include.

RECIPE GRAMMAR
           recipe ::= {<recipe-item>}
           recipe-item ::= <option> | <parameter> | <extend>
           option ::= (option <atom> {<atom>})
           parameter ::= (parameter <atom> <atom> <atom>)
           extend ::= (extend <atom>)
```

And here comes the description of the `--recipe` command line
parameter:

```
       --recipe=VAL
           Load the specified recipe. The  parameter specifies the name of the
           recipe along with an optional colon separated list of arguments.
           The  parameter has the following syntax: <name> : <arg1>=<val1> :
           <arg2>=<val2> : .... The <name> part could be either a path to a
           valid recipe or the name of a recipe, that would be search in the
           recipe search paths. A name may omit .recipe or .scm extensions for
           brevity. The valid representations of a recipe is either the recipe
           file itself (i.e., a file consisting of a list of s-expressions), a
           directory with a valid undefined file, or zip file, that contains a
           valid recipe directory. See the RECIPES section for more
           information. If a recipe is a zip file then it will be unpacked to
           a temporary folder that will be removed after the program
           terminates. Otherwise, all recipe resources will be used as is and
           won't be restored to their previous state, e.g., if the
           input/output files were provided, and they were modified, the
           modifications will persist.

```

This is how the `--list-recipes` option works:

```
$ bap --list-recipe
uaf                              Performs Use-After-Free Analysis
fuzz                             Testing Recipe
derive-inline                    Description was not provided
direct                           Testing Recipe
```

And this is how a description of the recipe is printed:

```
$ bap --show-recipe uaf
DESCRIPTION

Performs Use-After-Free Analysis

Uses the Primus promiscuous mode and fuzzes a binary through the specified
$entry points with the maximum depth set to $depth RTL instructions. Outputs
all found Use-After-Free incidents into the `incident` file.

PARAMETERS

- depth - maximum number of instructions per path (defaults to 8192)
- width - maximum number of visits to the same block (defaults to 128)
- entry - entry points (defaults to all-subroutines)

COMMAND LINE

--run \
--run-entry-points=all-subroutines \
--primus-greedy-scheduler \
--primus-limit-max-length=8192 \
--primus-limit-max-visited=128 \
--primus-promiscuous-mode \
--primus-lisp-load=posix,memcheck-malloc,limit-malloc \
--primus-print-obs=incident,incident-location,call-return \
--primus-print-rules=/tmp/recipe-943c0231-491a-43a5-86de-e6c2c662396f.unzipped/select.scm \
--primus-print-output=incidents \
--primus-lisp-channel-redirect=<stdin>:/tmp/recipe-943c0231-491a-43a5-86de-e6c2c662396f.unzipped/stdin,<stdout>:/tmp/recipe-943c0231-491a-43a5-86de-e6c2c662396f.unzipped/stdout \
--report-progress \
--log-dir=log
```
  • Loading branch information
ivg authored Apr 9, 2018
1 parent 878e39c commit 293ad10
Show file tree
Hide file tree
Showing 6 changed files with 260 additions and 64 deletions.
2 changes: 2 additions & 0 deletions src/.merlin
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
REC
PKG cmdliner
PKG curl
PKG parsexp
PKG uuidm
B ../_build/src
B ../_build/lib/bap_byteweight
107 changes: 106 additions & 1 deletion src/bap_cmdline_terms.ml
Original file line number Diff line number Diff line change
Expand Up @@ -200,8 +200,113 @@ let no_auto_load, no_auto_load_doc =
Arg.(value & flag & info ["disable-autoload"] ~doc), doc


let recipe_doc = [
`S "RECIPE DESCRIPTION";
`P "A recipe is either a single file or a directory (optionally
zipped) that contains a parametrized specification of command
line parameters and support files if necessary.";

`P
"The purpose of the recipe is to make bap runs reproducible, so
that you can share one simple file - the recipe, instead of a
bunch of scripts together with some verbal instructions. Since
recipes introduce an extra layer of abstraction they actually
simplify interaction with bap by hiding unnecessary
details. Recipes also provide a mechanism for building ready
solutions and sharing them with users.";

`P
"To use a recipe, just specify its name using the $(b,--recipe)
command line parameter. If a recipe has parameters then they could
be specified as colon separated list of $(b,<key>=<value>)
pairs. See the $(b,--recipe) parameter for more information. To
read the recipe description, including the list of parameters and
resulting command line, use the $(b,--show-recipe) command line
option. To list all installed recipes use the $(b,--list-recipes)
option.";

`P
"The main (and the only necessary) part of a recipe is the recipe
specification, that is a file that contains a list of recipe items
in an arbitrary order. Each item is either a command line
option, a parameter, or a reference to another recipe. All items
share the same syntax - they are flat s-expressions, i.e., a
whitespace separated list of strings enclosed in parentheses. The
first string in the lists denotes the type of the item. E.g.,
$(b,(option run-entry-points malloc calloc free)).";

`P
"The $(b,option) command requires one mandatory parameter, the
option name, and an arbitrary number of arguments that will be
passed to the corresponding command line option. If there are more
than one argument then they will be concatenated with the comman
symbol, e.g., $(b,(option opt a b c d)) will be translated to
$(b,--opt=a,b,c,d). Option arguments may contain substitution
symbols. A subsitution symbol starts with the dollar sign, that is
followed by a named (optionally delimited with curly braces, to
disambiguate it from the rest of the argument). There is one built
in parameter called $(b,prefix), that is substituted with the path
to the recipe top folder. See the $(b,parameter) command to learn
how to introduce parameters.";

`P
"The parameter command introduces a parameter to the recipe, i.e.,
a variable ingredient that could be changed when the recipe is
used. The parameter command has 3 arguments, all required. The
first argument is the parameter name, the second is the default
value, that is used if the a parameter wasn't set, and the last
argument is the parameter description. The substitution symbol
will be replaced with the default value of a parameter, if a value
of the parameter wasn't passed through the command line. Example,";

`Pre {|
(parameter depth 128 "maximum depth of analysis")
(option analysis-depth $depth)
|};

`P "
If the parameter is not set through the command line, then it will
be substituted with $(b,128) otherwise it will receive whatever
value a user has passed.
Finally, the $(b,extend) command is like the include statement in
the C preprocessor as it includes all the ingredients from another
recipe. (Make sure that you're not introducing loops!). The
command has one mandatory argument, the name of the recipe to
include.";
`S "RECIPE GRAMMAR";
`Pre "
recipe ::= {<recipe-item>}
recipe-item ::= <option> | <parameter> | <extend>
option ::= (option <atom> {<atom>})
parameter ::= (parameter <atom> <atom> <atom>)
extend ::= (extend <atom>)
"
]

let recipe =
let doc = "Load the specified recipe" in
let doc =
"Load the specified recipe. The $(docv) parameter specifies the
name of the recipe along with an optional colon separated list of
arguments. The $(docv) parameter has the following syntax:
$(b,<name> : <arg1>=<val1> : <arg2>=<val2> : ...).
The $(b,<name>) part could be either a path to a valid
recipe or the name of a recipe, that would be search in the recipe
search paths. A name may omit $(b,.recipe) or $(b,.scm) extensions
for brevity.
The valid representations of a recipe is either the recipe file
itself (i.e., a file consisting of a list of s-expressions), a
directory with a valid $(recipe.scm) file, or zip file, that
contains a valid recipe directory. See the $(b,RECIPES) section for
more information.
If a recipe is a zip file then it will be unpacked to a temporary
folder that will be removed after the program terminates. Otherwise,
all recipe resources will be used as is and won't be restored to
their previous state, e.g., if the input/output files were provided,
and they were modified, the modifications will persist." in
Arg.(value & opt (some string) None & info ["recipe"] ~doc)

let loader_options = [
Expand Down
1 change: 1 addition & 0 deletions src/bap_cmdline_terms.mli
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,4 @@ val recipe : string option Term.t
val loader_options : string list
val common_loader_options : Manpage.block list
val options_for_passes : Manpage.block list
val recipe_doc : Manpage.block list
17 changes: 13 additions & 4 deletions src/bap_main.ml
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,7 @@ let program_info =
`I ("$(b,--list-formats)", Bap_cmdline_terms.list_formats_doc)
] @ Bap_cmdline_terms.common_loader_options
@ Bap_cmdline_terms.options_for_passes
@ Bap_cmdline_terms.recipe_doc
@ [
`S "BUGS";
`P "Report bugs to \
Expand Down Expand Up @@ -260,9 +261,16 @@ let eval_recipe name =


let print_recipe r =
printf "%s@\nRecipe Arguments: %s@\n"
(Recipe.descr r)
(String.concat_array ~sep:" " (Recipe.argv r))
printf "DESCRIPTION@\n@\n%s@\n@\n" (Recipe.descr r);
let params = Recipe.params r in
if params <> [] then begin
printf "PARAMETERS@\n@\n";
List.iter params ~f:(printf "- %a@\n" Recipe.pp_param);
printf "@\n";
end;
let args = Recipe.argv r in
let sep = if Array.length args > 4 then " \\\n" else " " in
printf "COMMAND LINE@\n@\n%s@\n" (String.concat_array ~sep args)

let summary str =
match String.index str '\n' with
Expand All @@ -281,7 +289,8 @@ let print_recipes_and_exit () =
let name = Filename.chop_suffix file ".recipe" in
match Recipe.load ~paths:recipe_paths name with
| Ok r ->
printf "%-32s %s\n" name (summary (Recipe.descr r));
printf "%-32s %s\n" (Filename.basename name)
(summary (Recipe.descr r));
Recipe.cleanup r
| Error err ->
eprintf "Malformed recipe %s: %a@\n%!" file
Expand Down
Loading

0 comments on commit 293ad10

Please sign in to comment.