Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow to pass more information to ppx rewriters #2106

Merged
10 commits merged into from
May 11, 2019
Merged

Allow to pass more information to ppx rewriters #2106

10 commits merged into from
May 11, 2019

Conversation

mlasson
Copy link
Collaborator

@mlasson mlasson commented Apr 30, 2019 •
edited
Loading

This is a draft pull request that implements #2020.

Some design choices, I made:

  1. The equality of "cookie" request is tested after expansion of variable.

TODO:

  • : Non-regression test for various error messages,
  • : Forbid the feature for version below 1.10,
  • : Implements a sanity check for cookies name,
  • : Make the "library-name" cookie an instance of this feature,
  • : Double check escaping rules,
  • : Finally squash/rebase and sign the DCO.

Note: this contribution was initiated by @diml

ghost
ghost reviewed May 1, 2019
View reviewed changes
src/gen_meta.ml Outdated Show resolved Hide resolved
ghost
ghost reviewed May 1, 2019
View reviewed changes
src/gen_meta.ml Outdated Show resolved Hide resolved
ghost
ghost reviewed May 1, 2019
View reviewed changes
src/preprocessing.ml Outdated Show resolved Hide resolved
@ghost
Copy link

ghost commented May 1, 2019

Cool, thanks for looking at this! The escaping looks good to me BTW

mlasson
mlasson commented May 5, 2019
View reviewed changes
src/preprocessing.ml Outdated Show resolved Hide resolved
@mlasson mlasson force-pushed the ppx-cookies branch 4 times, most recently from 1406572 to 246289c Compare May 8, 2019 10:21
@mlasson
Copy link
Collaborator Author

mlasson commented May 8, 2019

I do not check anything about the sanity of the name. Is there a specification somewhere of what driver are supposed to implement to parse the "cookie argument" of the shape "name=value" ?

They just split on = (from left to right) the lhs is the name and the value is send the ocaml parser ?
So, should we check that the names do not contain the '=' character ?
I tried to do it, but I did not find the right place to place the error/warning; so I finally decided not to do anything.

@mlasson mlasson marked this pull request as ready for review May 8, 2019 11:19
@mlasson mlasson closed this May 8, 2019
@mlasson mlasson reopened this May 8, 2019
@ghost
Copy link

ghost commented May 8, 2019

There is no spec, but yh the drivers effectively just split on the first =. Checking that the name contain no equal sign seems good. We could also check for \x00 since that's not allowed on the command line. What about doing it when we parse the cookie name in lib_kind.ml? There is a plain_string function in Dune_lang.Decoder to conveniently parse strings that must satisfy a predicate.

ghost
ghost suggested changes May 8, 2019
View reviewed changes
doc/dune-files.rst Outdated Show resolved Hide resolved
src/preprocessing.ml Outdated Show resolved Hide resolved
src/preprocessing.ml Outdated Show resolved Hide resolved
src/preprocessing.ml Outdated Show resolved Hide resolved
src/preprocessing.ml Outdated Show resolved Hide resolved
src/preprocessing.ml Outdated Show resolved Hide resolved
src/preprocessing.ml Outdated Show resolved Hide resolved
ghost
ghost reviewed May 8, 2019
View reviewed changes
test/blackbox-tests/test-cases/dune-ppx-driver-system/driver-tests/dune
(modules ())
(libraries driver_print_args))

(env (_ (env-vars (ITALY "Biscotti") (FRANCE "Petit Beurre") (AMERICA "Oreo") (ENGLAND "Snickerdoodle"))))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haha 😆

@mlasson mlasson closed this May 8, 2019
@mlasson mlasson reopened this May 8, 2019
jeremiedimino and others added 6 commits May 9, 2019 15:44
@jeremiedimino @mlasson
Start support for passing extra cookies to ppx rewriters
018416f 
Signed-off-by: Jeremie Dimino <jeremie@dimino.org>
@mlasson
New feature: cookies in ppx_rewriter/deriver
2218937 
Signed-off-by: Marc Lasson <marc.lasson@lexifi.com>
@mlasson
Unit tests for lib_kind encode/decode
b9cbf96 
Signed-off-by: Marc Lasson <marc.lasson@lexifi.com>
@mlasson
Update CHANGES
c52ee1e 
Signed-off-by: Marc Lasson <marc.lasson@lexifi.com>
@mlasson
Update doc
eff295a 
Signed-off-by: Marc Lasson <marc.lasson@lexifi.com>
@mlasson
Forbid cookie name with '='
40c61fd 
Signed-off-by: Marc Lasson <marc.lasson@lexifi.com>
@mlasson
Implement reviewer's suggestions
dae0bdd 
Signed-off-by: Marc Lasson <marc.lasson@lexifi.com>
@mlasson
Remove lib_kind unit test
4551f0f 
Signed-off-by: Marc Lasson <marc.lasson@lexifi.com>
mlasson and others added 2 commits May 9, 2019 18:36
@mlasson
Fix accidental regression
5e6bff4 
Signed-off-by: Marc Lasson <marc.lasson@lexifi.com>
@jeremiedimino
Merge branch 'master' into ppx-cookies
2ac68aa
@ghost ghost merged commit 434bd16 into ocaml:master May 11, 2019
@ghost
Copy link

ghost commented May 11, 2019

Looks good, thanks a lot for this contribution!

rgrinberg added a commit to rgrinberg/opam-repository that referenced this pull request May 30, 2019
@rgrinberg
[new release] dune (1.10.0)
31f7a06 
CHANGES:

- Restricted the set of variables available for expansion in the destination
  filename of `install` stanza to simplify implementation and avoid dependency
  cycles. (ocaml/dune#2073, @aalekseyev, @diml)

- [menhir] call menhir from context root build_dir (ocaml/dune#2067, @ejgallego,
  review by @diml, @rgrinberg)

- [coq] Add `coq.pp` stanza to help with pre-processing of grammar
  files (ocaml/dune#2054, @ejgallego, review by @rgrinberg)

- Add a new more generic form for the *promote* mode: `(promote
  (until-clean) (into <dir>))` (ocaml/dune#2068, @diml)

- Allow to promote only a subset of the targets via `(promote (only
  <pred>))`. For instance: `(promote (only *.mli))` (ocaml/dune#2068, @diml)

- Improve the behavior when a strict subset of the targets of a rule is already
  in the source tree for projects using the dune language < 1.10 (ocaml/dune#2068, fixes
  ocaml/dune#2061, @diml)

- With lang dune >= 1.10, rules in standard mode are no longer allowed to
  produce targets that are present in the source tree. This has been a warning
  for long enough (ocaml/dune#2068, @diml)

- Allow %{...} variables in pps flags (ocaml/dune#2076, @mlasson review by @diml and
  @aalekseyev).

- Add a 'cookies' option to ppx_rewriter/deriver flags in library stanzas. This
  allow to specify cookie requests from variables expanded at each invocation of
  the preprocessor. (ocaml/dune#2106, @mlasson @diml)

- Add more opam metadata and use it to generate `.opam` files. In particular, a
  `package` field has been added to specify package specific information.
  (ocaml/dune#2017, ocaml/dune#2091, @avsm, @jonludlam, @rgrinberg)

- Clean up the special support for `findlib.dynload`. Before, Dune would simply
  match on the library name. Now, we only match on the findlib package name when
  the library doesn't come from Dune. Someone writing a library called
  `findlib.dynload` with Dune would have to add `(special_builtin_support
  findlib_dynload)` to trigger the special behavior. (ocaml/dune#2115, @diml)

- Install the `future_syntax` preprocessor as `ocaml-syntax-shims.exe` (ocaml/dune#2125,
  @rgrinberg)

- Hide full command on errors and warnings in development and show them in CI.
  (detected using the `CI` environment variable). Commands for which the
  invocation might be omitted must output an error prefixed with `File `. Add an
  `--always-show-command-line` option to disable this behavior and always show
  the full command. (ocaml/dune#2120, fixes ocaml/dune#1733, @rgrinberg)

- In `dune-workspace` files, add the ability to choose the host context and to
  create duplicates of the default context with different settings. (ocaml/dune#2098,
  @TheLortex, review by @diml, @rgrinberg and @aalekseyev)

- Add support for hg in `dune subst` (ocaml/dune#2135, @diml)

- Don't build documentation for implementations of virtual libraries (ocaml/dune#2141,
  fixes ocaml/dune#2138, @jonludlam)

- Fix generation of the `-pp` flag in .merlin (ocaml/dune#2142, @rgrinberg)

- Make `dune subst` add a `(version ...)` field to the `dune-project`
  file (ocaml/dune#2148, @diml)

- Add the `%{os_type}` variable, which is a short-hand for
  `%{ocaml-config:os_type}` (ocaml/dune#1764, @diml)

- Allow `enabled_if` fields in `library` stanzas, restricted to the
  `%{os_type}`, `%{model}`, `%{architecture}`, `%{system}` variables (ocaml/dune#1764,
  ocaml/dune#2164 @diml, @rgrinberg)

- Fix `chdir` on external and source paths. Dune will also fail gracefully if
  the external or source path does not exist (ocaml/dune#2165, fixes ocaml/dune#2158, @rgrinberg)

- Support the `.cc` extension fro C++ sources (ocaml/dune#2195, fixes ocaml/dune#83, @rgrinberg)

- Run `ocamlformat` relative to the context root. This improves the locations of
  errors. (ocaml/dune#2196, fixes ocaml/dune#1370, @rgrinberg)

- Fix detection of `README`, `LICENSE`, `CHANGE`, and `HISTORY` files. These
  would be undetected whenever the project was nested in another workspace.
  (ocaml/dune#2194, @rgrinberg)

- Fix generation of `.merlin` whenever there's more than one stanza with the
  same ppx preprocessing specification (ocaml/dune#2209 ,fixes ocaml/dune#2206, @rgrinberg)

- Fix generation of `.meriln` in the presence of the `copy_files` stanza and
  preprocessing specifications of other stanazs. (ocaml/dune#2211, fixes ocaml/dune#2206,
  @rgrinberg)
@rgrinberg rgrinberg mentioned this pull request May 30, 2019
Closed
rgrinberg added a commit to rgrinberg/opam-repository that referenced this pull request Jun 4, 2019
@rgrinberg
[new release] dune (1.10.0)
cfa54d7 
CHANGES:

- Restricted the set of variables available for expansion in the destination
  filename of `install` stanza to simplify implementation and avoid dependency
  cycles. (ocaml/dune#2073, @aalekseyev, @diml)

- [menhir] call menhir from context root build_dir (ocaml/dune#2067, @ejgallego,
  review by @diml, @rgrinberg)

- [coq] Add `coq.pp` stanza to help with pre-processing of grammar
  files (ocaml/dune#2054, @ejgallego, review by @rgrinberg)

- Add a new more generic form for the *promote* mode: `(promote
  (until-clean) (into <dir>))` (ocaml/dune#2068, @diml)

- Allow to promote only a subset of the targets via `(promote (only
  <pred>))`. For instance: `(promote (only *.mli))` (ocaml/dune#2068, @diml)

- Improve the behavior when a strict subset of the targets of a rule is already
  in the source tree for projects using the dune language < 1.10 (ocaml/dune#2068, fixes
  ocaml/dune#2061, @diml)

- With lang dune >= 1.10, rules in standard mode are no longer allowed to
  produce targets that are present in the source tree. This has been a warning
  for long enough (ocaml/dune#2068, @diml)

- Allow %{...} variables in pps flags (ocaml/dune#2076, @mlasson review by @diml and
  @aalekseyev).

- Add a 'cookies' option to ppx_rewriter/deriver flags in library stanzas. This
  allow to specify cookie requests from variables expanded at each invocation of
  the preprocessor. (ocaml/dune#2106, @mlasson @diml)

- Add more opam metadata and use it to generate `.opam` files. In particular, a
  `package` field has been added to specify package specific information.
  (ocaml/dune#2017, ocaml/dune#2091, @avsm, @jonludlam, @rgrinberg)

- Clean up the special support for `findlib.dynload`. Before, Dune would simply
  match on the library name. Now, we only match on the findlib package name when
  the library doesn't come from Dune. Someone writing a library called
  `findlib.dynload` with Dune would have to add `(special_builtin_support
  findlib_dynload)` to trigger the special behavior. (ocaml/dune#2115, @diml)

- Install the `future_syntax` preprocessor as `ocaml-syntax-shims.exe` (ocaml/dune#2125,
  @rgrinberg)

- Hide full command on errors and warnings in development and show them in CI.
  (detected using the `CI` environment variable). Commands for which the
  invocation might be omitted must output an error prefixed with `File `. Add an
  `--always-show-command-line` option to disable this behavior and always show
  the full command. (ocaml/dune#2120, fixes ocaml/dune#1733, @rgrinberg)

- In `dune-workspace` files, add the ability to choose the host context and to
  create duplicates of the default context with different settings. (ocaml/dune#2098,
  @TheLortex, review by @diml, @rgrinberg and @aalekseyev)

- Add support for hg in `dune subst` (ocaml/dune#2135, @diml)

- Don't build documentation for implementations of virtual libraries (ocaml/dune#2141,
  fixes ocaml/dune#2138, @jonludlam)

- Fix generation of the `-pp` flag in .merlin (ocaml/dune#2142, @rgrinberg)

- Make `dune subst` add a `(version ...)` field to the `dune-project`
  file (ocaml/dune#2148, @diml)

- Add the `%{os_type}` variable, which is a short-hand for
  `%{ocaml-config:os_type}` (ocaml/dune#1764, @diml)

- Allow `enabled_if` fields in `library` stanzas, restricted to the
  `%{os_type}`, `%{model}`, `%{architecture}`, `%{system}` variables (ocaml/dune#1764,
  ocaml/dune#2164 @diml, @rgrinberg)

- Fix `chdir` on external and source paths. Dune will also fail gracefully if
  the external or source path does not exist (ocaml/dune#2165, fixes ocaml/dune#2158, @rgrinberg)

- Support the `.cc` extension fro C++ sources (ocaml/dune#2195, fixes ocaml/dune#83, @rgrinberg)

- Run `ocamlformat` relative to the context root. This improves the locations of
  errors. (ocaml/dune#2196, fixes ocaml/dune#1370, @rgrinberg)

- Fix detection of `README`, `LICENSE`, `CHANGE`, and `HISTORY` files. These
  would be undetected whenever the project was nested in another workspace.
  (ocaml/dune#2194, @rgrinberg)

- Fix generation of `.merlin` whenever there's more than one stanza with the
  same ppx preprocessing specification (ocaml/dune#2209 ,fixes ocaml/dune#2206, @rgrinberg)

- Fix generation of `.merlin` in the presence of the `copy_files` stanza and
  preprocessing specifications of other stanazs. (ocaml/dune#2211, fixes ocaml/dune#2206,
  @rgrinberg)

- Run `refmt` from the context's root directory. This improves error messages in
  case of syntax errors. (ocaml/dune#2223, @rgrinberg)

- In .merlin files, don't pass `-dump-ast` to the `future_syntax` preprocessor.
  Merlin doesn't seem to like it when binary AST is generated by a `-pp`
  preprocessor. (ocaml/dune#2236, @aalekseyev)

- `dune install` will verify that all files mentioned in all .install files
  exist before trying to install anything. This prevents partial installation of
  packages (ocaml/dune#2230, @rgrinberg)
@rgrinberg rgrinberg mentioned this pull request Jun 4, 2019
Merged
mlasson added a commit to mlasson/dune that referenced this pull request Jul 19, 2019
@mlasson
Allow to pass more information via cookies to ppx rewriters (ocaml#2106)
1ef68ac 
Signed-off-by: Marc Lasson <marc.lasson@lexifi.com>
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aalekseyev aalekseyev Awaiting requested review from aalekseyev

@emillon emillon Awaiting requested review from emillon

@rgrinberg rgrinberg Awaiting requested review from rgrinberg

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mlasson @jeremiedimino