Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Map (re_export) to exports field in META files #10831

Merged
merged 9 commits into from
Aug 19, 2024
Merged

Map (re_export) to exports field in META files #10831

merged 9 commits into from
Aug 19, 2024

Conversation

nojb
Copy link
Collaborator

@nojb nojb commented Aug 17, 2024 •
edited
Loading

Fixes #10830

See #10830 and the links there (especially https://discuss.ocaml.org/t/proposal-a-new-exports-field-in-findlib-meta-files/13947) for the context. Each commit contains a corresponding test.

  • The first commit makes Dune output an exports field in the generated META containing any library dependencies which appear in (re_export).
  • The second commit makes Dune interpret exports field in META files by mapping the libraries listed in it to (re_export). Note that exports is semantically interpreted as a list of boolean flags on the existing requires field. This means: libraries listed in exports but not in requires are ignored, and exports is assumed not to be "predicated" (in the Findlib sense).
  • The third commit adapts (deprecated_library_name) to use (exports) (this is actually one of the motivating use-cases mentioned in https://discuss.ocaml.org/t/proposal-a-new-exports-field-in-findlib-meta-files/13947)

nojb added 3 commits August 17, 2024 08:23
@nojb
Generate exports field in META file for (re_export) libraries
60c8388 
Signed-off-by: Nicolás Ojeda Bär <n.oje.bar@gmail.com>
@nojb
Interpret exports field in META files
1bdd62a 
Signed-off-by: Nicolás Ojeda Bär <n.oje.bar@gmail.com>
@nojb
Use exports field in META generated by (deprecated_library_name)
c90f9f0 
Signed-off-by: Nicolás Ojeda Bär <n.oje.bar@gmail.com>
@nojb nojb mentioned this pull request Aug 17, 2024
Closed
@nojb
Copy link
Collaborator Author

nojb commented Aug 17, 2024

  • exports is assumed not to be "predicated" (in the Findlib sense).

Actually, I am not sure about this point. Maybe we should interpret predicates in the new exports field.

@rgrinberg
Copy link
Member

I think you need to guard the Meta generation behind a 3.17 check.

@nojb
Guard META generation of exports with version >= (3, 17)
c2cdbb3 
Signed-off-by: Nicolás Ojeda Bär <n.oje.bar@gmail.com>
@nojb
Copy link
Collaborator Author

nojb commented Aug 17, 2024

I think you need to guard the Meta generation behind a 3.17 check.

Done in c2cdbb3. Thanks!

@rgrinberg
Copy link
Member

Are there any tests for how we interpret this field from META files? I see how we generate them, but we should test how they're read (whenever there's no dune-package file)

@rgrinberg rgrinberg added this to the 3.17.0 milestone Aug 17, 2024
@nojb
Copy link
Collaborator Author

nojb commented Aug 17, 2024

Are there any tests for how we interpret this field from META files? I see how we generate them, but we should test how they're read (whenever there's no dune-package file)

Yes, there is one expect test: https://github.com/ocaml/dune/pull/10831/files#diff-b575837081f645adcc29f1302fc07bb1b2044801712369ccbe094aea0e41f97b

@rgrinberg
Copy link
Member

I see a unit test that we parse them correctly. But it would be nice to demonstrate that we can actually build against something that such a META. This isn't strictly necessary, but the only tangible benefit this feature provides to dune users is the ability to use Daniel's libraries correctly. So I do think this is the most important criteria for us.

@nojb
Copy link
Collaborator Author

nojb commented Aug 18, 2024

it would be nice to demonstrate that we can actually build against something that such a META

OK, I will add a test showing this later today. Thanks!

@nojb
Add compilation test for the new META field
7dc7c8c 
Signed-off-by: Nicolás Ojeda Bär <n.oje.bar@gmail.com>
@nojb
Copy link
Collaborator Author

nojb commented Aug 18, 2024

it would be nice to demonstrate that we can actually build against something that such a META

OK, I will add a test showing this later today. Thanks!

I pushed a new dedicated test meta-exports.t that does this now: https://github.com/ocaml/dune/pull/10831/files#diff-0f4411a3b78fcb4907a6dd8530f238a7593c81dbf0bd1bb9069e9cb117a4f4f3. Since the new test checks both the ability to consume and generate the exports fields, I reverted the changes to the existing test meta-gen.t.

nojb added 2 commits August 19, 2024 07:00
@nojb
Changes
09102f7 
Signed-off-by: Nicolás Ojeda Bär <n.oje.bar@gmail.com>
@nojb
Merge branch 'main' into META_exports
27413f7
@nojb
Copy link
Collaborator Author

nojb commented Aug 19, 2024

Thanks for the approval @rgrinberg. I pushed a Changelog entry in preparation for merging.

@dbuenzli: does the PR look OK from your side? You can take a look at the CRAM test meta-exports.t to get an idea of what was implemented.

@dbuenzli
Copy link

@dbuenzli: does the PR look OK from your side? You can take a look at the CRAM test meta-exports.t to get an idea of what was implemented.

I did that. I would say yes.

@nojb
Copy link
Collaborator Author

nojb commented Aug 19, 2024

@dbuenzli: does the PR look OK from your side? You can take a look at the CRAM test meta-exports.t to get an idea of what was implemented.

I did that. I would say yes.

Thanks for the confirmation!

@nojb nojb merged commit 2c811fe into ocaml:main Aug 19, 2024
26 of 28 checks passed
@nojb nojb deleted the META_exports branch August 19, 2024 20:32
anmonteiro pushed a commit to anmonteiro/dune that referenced this pull request Nov 17, 2024
@nojb @anmonteiro
Map (re_export) to exports field in META files (ocaml#10831)
0479f0d 
Signed-off-by: Nicolás Ojeda Bär <n.oje.bar@gmail.com>
maiste added a commit to maiste/opam-repository that referenced this pull request Nov 25, 2024
@maiste
[new release] dune (15 packages) (3.17.0~alpha0)
f1d49be 
CHANGES:

### Fixed

- Show the context name for errors happening in non-default contexts.
  (ocaml/dune#10414, fixes ocaml/dune#10378, @jchavarri)

- Correctly declare dependencies of indexes so that they are rebuilt when
  needed. (ocaml/dune#10623, @voodoos)

- Don't depend on coq-stdlib being installed when expanding variables
  of the `coq.version` family (ocaml/dune#10631, fixes ocaml/dune#10629, @gares)

- Error out if no files are found when using `copy_files`. (ocaml/dune#10649, @jchavarri)

- Re_export dune-section private library in the dune-site library stanza,
  in order to avoid failure when generating and building sites modules
  with implicit_transitive_deps = false. (ocaml/dune#10650, fixes ocaml/dune#9661, @MA0100)

- Expect test fixes: support multiple modes and fix dependencies when there is
  a custom runner (ocaml/dune#10671, @vouillon)

- In a `(library)` stanza with `(extra_objects)` and `(foreign_stubs)`, avoid
  double linking the extra object files in the final executable.
  (ocaml/dune#10783, fixes ocaml/dune#10785, @nojb)

- Map `(re_export)` library dependencies to the `exports` field in `META` files,
  and vice-versa. This field was proposed in to
  https://discuss.ocaml.org/t/proposal-a-new-exports-field-in-findlib-meta-files/13947.
  The field is included in Dune-generated `META` files only when the Dune lang
  version is >= 3.17.
  (ocaml/dune#10831, fixes ocaml/dune#10830, @nojb)

- Fix staged pps preprocessors on Windows (which were not working at all
  previously) (ocaml/dune#10869, fixes ocaml/dune#10867, @nojb)

- Fix `dune describe` when an executable is disabled with `enabled_if`.
  (ocaml/dune#10881, fixes ocaml/dune#10779, @moyodiallo)

- Fix an issue where C stubs would be rebuilt whenever the stderr of Dune was
  redirected. (ocaml/dune#10883, fixes ocaml/dune#10882, @nojb)

- Format long lists in s-expressions to fill the line instead of formatting
  them in a vertical way (ocaml/dune#10892, fixes ocaml/dune#10860, @nojb)

- Fix the URL opened by the command `dune ocaml doc`. (ocaml/dune#10897, @gridbugs)

- Fix the file referred to in the error/warning message displayed due to the
  dune configuration version not supporting a particular configuration
  stanza in use. (ocaml/dune#10923, @H-ANSEN)

- Fix `enabled_if` when it uses `env` variable. (ocaml/dune#10936, fixes ocaml/dune#10905, @moyodiallo)

- Fix exec -w for relative paths with --root argument (ocaml/dune#10982, @gridbugs)

- Do not ignore the `(locks ..)` field in the `test` and `tests` stanza
  (ocaml/dune#11081, @rgrinberg)

- Tolerate files without extension when generating merlin rules.
  (ocaml/dune#11128, @anmonteiro)

### Added

- Make Merlin/OCaml-LSP aware of "hidden" dependencies used by
  `(implicit_transitive_deps false)` via the `-H` compiler flag. (ocaml/dune#10535, @voodoos)

- Add support for the -H flag (introduced in OCaml compiler 5.2) in dune
  (requires lang versions 3.17). This adaptation gives
  the correct semantics for `(implicit_transitive_deps false)`.
  (ocaml/dune#10644, fixes ocaml/dune#9333, ocsigen/tyxml#274, ocaml/dune#2733, ocaml/dune#4963, @MA0100)

- Add support for specifying Gitlab organization repositories in `source`
  stanzas (ocaml/dune#10766, fixes ocaml/dune#6723, @H-ANSEN)

- New option to control jsoo sourcemap generation in env and executable stanza
  (ocaml/dune#10777, fixes ocaml/dune#10673, @hhugo)

- One can now control jsoo compilation_mode inside an executable stanza
  (ocaml/dune#10777, fixes ocaml/dune#10673, @hhugo)

- Add support for specifying default values of the `authors`, `maintainers`, and
  `license` stanzas of the `dune-project` file via the dune config file. Default
  values are set using the `(project_defaults)` stanza (ocaml/dune#10835, @H-ANSEN)

- Add names to source tree events in performance traces (ocaml/dune#10884, @jchavarri)

- Add `codeberg` as an option for defining project sources in dune-project
  files. For example, `(source (codeberg user/repo))`. (ocaml/dune#10904, @nlordell)

- `dune runtest` can now run individual tests with `dune runtest mytest.t`
  (ocaml/dune#11041, @Alizter).

- Wasm_of_ocaml support (ocaml/dune#11093, @vouillon)

- Add a `coqdep_flags` field to the `coq` field of the `env` stanza, and to the `coq.theory` stanza, allowing to configure `coqdep` flags.
  (ocaml/dune#11094, @rlepigre)

### Changed

- Remove all remnants of the experimental `patch-back-source-tree`. (ocaml/dune#10771,
  @rgrinberg)

- Change the preset value for author and maintainer fields in the
  `dune-project` file to encourage including emails. (ocaml/dune#10848, @punchagan)

- Tweak the preset value for tags in the `dune-project` file to hint at topics
  not having a special meaning. (ocaml/dune#10849, @punchagan)

- Change some colors to improve readability in light-mode terminals
  (ocaml/dune#10890, @gridbugs)

- Forward the linkall flag to jsoo in whole program compilation as well (ocaml/dune#10935, @hhugo)

- Configurator uses `pkgconf` as pkg-config implementation when available
  and forwards it the `target` of `ocamlc -config`. (ocaml/dune#10937, @pirbo)

- Enable Dune cache by default. Add a new Dune cache setting
  `enabled-except-user-rules`, which enables the Dune cache, but excludes
  user-written rules from it. This is a conservative choice that can avoid
  breaking rules whose dependencies are not correctly specified. This is the
  current default. (ocaml/dune#10944, ocaml/dune#10710, @nojb, @ElectreAAS)

- Do not add `dune` dependency in `dune-project` when creating projects with
  `dune init proj`. The Dune dependency is implicitely added when generating
  opam files (ocaml/dune#11129, @Leonidas-from-XIV)
@maiste maiste mentioned this pull request Nov 25, 2024
Closed
maiste added a commit to maiste/opam-repository that referenced this pull request Nov 27, 2024
@maiste
[new release] dune (15 packages) (3.17.0)
ca0426d 
CHANGES:

### Fixed

- Show the context name for errors happening in non-default contexts.
  (ocaml/dune#10414, fixes ocaml/dune#10378, @jchavarri)

- Correctly declare dependencies of indexes so that they are rebuilt when
  needed. (ocaml/dune#10623, @voodoos)

- Don't depend on coq-stdlib being installed when expanding variables
  of the `coq.version` family (ocaml/dune#10631, fixes ocaml/dune#10629, @gares)

- Error out if no files are found when using `copy_files`. (ocaml/dune#10649, @jchavarri)

- Re_export dune-section private library in the dune-site library stanza,
  in order to avoid failure when generating and building sites modules
  with implicit_transitive_deps = false. (ocaml/dune#10650, fixes ocaml/dune#9661, @MA0100)

- Expect test fixes: support multiple modes and fix dependencies when there is
  a custom runner (ocaml/dune#10671, @vouillon)

- In a `(library)` stanza with `(extra_objects)` and `(foreign_stubs)`, avoid
  double linking the extra object files in the final executable.
  (ocaml/dune#10783, fixes ocaml/dune#10785, @nojb)

- Map `(re_export)` library dependencies to the `exports` field in `META` files,
  and vice-versa. This field was proposed in to
  https://discuss.ocaml.org/t/proposal-a-new-exports-field-in-findlib-meta-files/13947.
  The field is included in Dune-generated `META` files only when the Dune lang
  version is >= 3.17.
  (ocaml/dune#10831, fixes ocaml/dune#10830, @nojb)

- Fix staged pps preprocessors on Windows (which were not working at all
  previously) (ocaml/dune#10869, fixes ocaml/dune#10867, @nojb)

- Fix `dune describe` when an executable is disabled with `enabled_if`.
  (ocaml/dune#10881, fixes ocaml/dune#10779, @moyodiallo)

- Fix an issue where C stubs would be rebuilt whenever the stderr of Dune was
  redirected. (ocaml/dune#10883, fixes ocaml/dune#10882, @nojb)

- Fix the URL opened by the command `dune ocaml doc`. (ocaml/dune#10897, @gridbugs)

- Fix the file referred to in the error/warning message displayed due to the
  dune configuration version not supporting a particular configuration
  stanza in use. (ocaml/dune#10923, @H-ANSEN)

- Fix `enabled_if` when it uses `env` variable. (ocaml/dune#10936, fixes ocaml/dune#10905, @moyodiallo)

- Fix exec -w for relative paths with --root argument (ocaml/dune#10982, @gridbugs)

- Do not ignore the `(locks ..)` field in the `test` and `tests` stanza
  (ocaml/dune#11081, @rgrinberg)

- Tolerate files without extension when generating merlin rules.
  (ocaml/dune#11128, @anmonteiro)

### Added

- Make Merlin/OCaml-LSP aware of "hidden" dependencies used by
  `(implicit_transitive_deps false)` via the `-H` compiler flag. (ocaml/dune#10535, @voodoos)

- Add support for the -H flag (introduced in OCaml compiler 5.2) in dune
  (requires lang versions 3.17). This adaptation gives
  the correct semantics for `(implicit_transitive_deps false)`.
  (ocaml/dune#10644, fixes ocaml/dune#9333, ocsigen/tyxml#274, ocaml/dune#2733, ocaml/dune#4963, @MA0100)

- Add support for specifying Gitlab organization repositories in `source`
  stanzas (ocaml/dune#10766, fixes ocaml/dune#6723, @H-ANSEN)

- New option to control jsoo sourcemap generation in env and executable stanza
  (ocaml/dune#10777, fixes ocaml/dune#10673, @hhugo)

- One can now control jsoo compilation_mode inside an executable stanza
  (ocaml/dune#10777, fixes ocaml/dune#10673, @hhugo)

- Add support for specifying default values of the `authors`, `maintainers`, and
  `license` stanzas of the `dune-project` file via the dune config file. Default
  values are set using the `(project_defaults)` stanza (ocaml/dune#10835, @H-ANSEN)

- Add names to source tree events in performance traces (ocaml/dune#10884, @jchavarri)

- Add `codeberg` as an option for defining project sources in dune-project
  files. For example, `(source (codeberg user/repo))`. (ocaml/dune#10904, @nlordell)

- `dune runtest` can now run individual tests with `dune runtest mytest.t`
  (ocaml/dune#11041, @Alizter).

- Wasm_of_ocaml support (ocaml/dune#11093, @vouillon)

- Add a `coqdep_flags` field to the `coq` field of the `env` stanza, and to the `coq.theory` stanza, allowing to configure `coqdep` flags.
  (ocaml/dune#11094, @rlepigre)

### Changed

- Remove all remnants of the experimental `patch-back-source-tree`. (ocaml/dune#10771,
  @rgrinberg)

- Change the preset value for author and maintainer fields in the
  `dune-project` file to encourage including emails. (ocaml/dune#10848, @punchagan)

- Tweak the preset value for tags in the `dune-project` file to hint at topics
  not having a special meaning. (ocaml/dune#10849, @punchagan)

- Change some colors to improve readability in light-mode terminals
  (ocaml/dune#10890, @gridbugs)

- Forward the linkall flag to jsoo in whole program compilation as well (ocaml/dune#10935, @hhugo)

- Configurator uses `pkgconf` as pkg-config implementation when available
  and forwards it the `target` of `ocamlc -config`. (ocaml/dune#10937, @pirbo)

- Enable Dune cache by default. Add a new Dune cache setting
  `enabled-except-user-rules`, which enables the Dune cache, but excludes
  user-written rules from it. This is a conservative choice that can avoid
  breaking rules whose dependencies are not correctly specified. This is the
  current default. (ocaml/dune#10944, ocaml/dune#10710, @nojb, @ElectreAAS)

- Do not add `dune` dependency in `dune-project` when creating projects with
  `dune init proj`. The Dune dependency is implicitely added when generating
  opam files (ocaml/dune#11129, @Leonidas-from-XIV)
@maiste maiste mentioned this pull request Nov 27, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rgrinberg rgrinberg rgrinberg approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
3.17.0
Development

Successfully merging this pull request may close these issues.

Map (re_export) to exports field in META files
3 participants
@nojb @rgrinberg @dbuenzli