[new release] dune-build-info and dune (1.11.0)

[rgrinberg]

Embed build informations inside executable

• Project page: https://github.com/ocaml/dune
• Documentation: https://dune.readthedocs.io/

CHANGES:

• Don't select all local implementations in dune utop. Instead, let the
  default implementation selection do its job. (In dune utop, stop selecting all local implementations  dune#2327, fixes dune utop fails with multiple virtual module implementations dune#2323, @TheLortex,
  review by @rgrinberg)

• Check that selected implementations (either by variants or default
  implementations) are indeed implementations. (Check user-written input of implementations dune#2328, @TheLortex, review by
  @rgrinberg)

• Don't reserve the Ppx toplevel module name for ppx rewriters (Generate _ppx.ml rather than ppx.ml dune#2242, @diml)

• Redesign of the library variant feature according to the [RFC] New variant proposal dune#2134 proposal. The
  set of variants is now computed when the virtual library is installed.
  Introducing a new external_variant stanza. (Variants v2 dune#2169, fixes [RFC] New variant proposal dune#2134, @TheLortex,
  review by @diml)

• Add proper line directives when copying .cc and .cxx sources (Unhardcode extensions dune#2275,
  @rgrinberg)

• Fix error message for missing C++ sources. The .cc extension was always
  ignored before. (Unhardcode extensions dune#2275, @rgrinberg)

• Add $ dune init project subcommand to create project boilerplate according
  to a common template. (Add dune init proj subcommand to initialize project skeletons dune#2185, fixes a wizard / quickstart a la oasis would be cool dune#159, @shonfeder)

• Allow to run inline tests in javascript with nodejs (Running inline tests with javascript/nodejs  dune#2266, @hhugo)

• Build ppx.exe as compiling host binary. (Set ppx.exe to be built for the compiling host. dune#2286, fixes Cross-compiling sedlex (ppx) for windows dune#2252, @toots, review
  by @rgrinberg and @diml)

• Add a cinaps extension and stanza for better integration with the
  cinaps tool tool (Add support for cinaps dune#2269,
  @diml)

• Allow to embed build info in executables such as version and list
  and version of statically linked libraries (Formalise the management of versions dune#2224, @diml)

• Set version in META and dune-package files to the one read from
  the vcs when no other version is available (Formalise the management of versions dune#2224, @diml)

• Add a variable %{target} to be used in situations where the context
  requires at most one word, so %{targets} can be confusing; stdout
  redirections and "-o" arguments of various tools are the main use
  case; also, introduce a separate field target that must be used
  instead of targets in those situations. (Add a variable %{target} dune#2341, @aalekseyev)

• Fix dependency graph of wrapped_compat modules. Previously, the dependency on
  the user written entry module was omitted. ([WIP] Unified module handling dune#2305, @rgrinberg)

• Allow to promote executables built with an executable stanza
  (Allow to promote executables to the source tree dune#2379, @diml)

• When instantiating an implementation with a variant, make sure it matches
  virtual library's list of known implementations. (Fix separate installation of implementations dune#2361, fixes separate installation of virtual-modules packages fails dune#2322,
  @TheLortex, review by @rgrinberg)

• Add a variable %{ignoring_promoted_rules} that is true when
  --ingore-promoted-rules is passed on the command line and false
  otherwise (Add ignoring_promoted_rules variable dune#2382, @diml)

• Fix a bug in future_syntax where the characters @ and & were
  not distinguished in the names of binding operators (let@ was the
  same as let&) (fix a typo in ocaml-syntax-shims dune#2376, @aalekseyev, @diml)

• Workspaces with non unique project names are now supported. (Relax restriction on project names being unique dune#2377, fix Internal error when 2 dune-project files have the same name stanza dune#2325,
  @rgrinberg)

• Improve opam generation to include the dune dependncies with the minimum
  constraint set based on the dune language version specified in the
  dune-project file. (2383, @avsm)

• The order of fields in the generated opam file now follows order preferred in
  opam-lib. (@avsm, do opam file generation in the same order as opam lint dune#2380)

• Fix coloring of error messages from the compiler (@diml, Fix coloring of messages from the compiler dune#2384)

• Add warning 66 to default set of warnings starting for dune projects with
  language verison >= 1.11 (@rgrinberg, @diml, fixes Set warnings set according to dune's language dune#2299)

• Add (dialect ...) stanza
  (@nojb, Add support for "dialects" dune#2404)

• Add a --context argument to dune install/uninstall (@diml, Add a --context argument to dune install/uninstall dune#2412)

• Do not warn about merlin files pre 1.9. This warning can only be disabled in
  1.9 (Silence merlin approximation warning in lang < 1.9 dune#2421, fixes merlin approximation warning is not always actionable dune#2399, @emillon)

[camelus]

☀️ All lint checks passed aa4bf16

• These packages passed lint tests: dune-build-info.1.11.0, dune.1.11.0

---

☀️ Installability check (+2)

• new installable packages (2): dune.1.11.0 dune-build-info.1.11.0

[avsm]

#14548 fixes some of the rev dep failures

[avsm]

dune-build-info fails:

#=== ERROR while compiling dune-build-info.1.11.0 =============================#
# context              2.0.4 | linux/x86_64 | ocaml-base-compiler.4.08.0 | git+file:///home/opam/opam-repository
# path                 ~/.opam/4.08/.opam-switch/build/dune-build-info.1.11.0
# command              ~/.opam/4.08/bin/dune build -p dune-build-info -j 2 @install @runtest
# exit-code            1
# env-file             ~/.opam/log/dune-build-info-17-d2c37b.env
# output-file          ~/.opam/log/dune-build-info-17-d2c37b.out
### output ###
# File "doc/dune", line 20, characters 0-134:
# 20 | (rule
# 21 |  (targets dune.inc.gen)
# 22 |  (deps    (package dune))
# 23 |  (action
# 24 |   (with-stdout-to %{targets}
# 25 |    (run bash %{dep:update-jbuild.sh}))))
# Error: No rule found for alias .dune-files
# File "test/expect-tests/dune", line 7, characters 12-18:
# 7 |  (libraries stdune dune wp_dune
#                 ^^^^^^
# Error: Library "stdune" not found.
# Hint: try: dune external-lib-deps --missing -p dune-build-info -j 2 @install
#   @runtest
# File "test/blackbox-tests/dune", line 6, characters 12-18:
# 6 |  (libraries stdune dune_re wp_dune test_common dune configurator))
#                 ^^^^^^
# Error: Library "stdune" not found.
# Hint: try: dune external-lib-deps --missing -p dune-build-info -j 2 @install
#   @runtest
# File "test/blackbox-tests/dune", line 12, characters 12-18:
# 12 |  (libraries stdune dune_lang)
#                  ^^^^^^
# Error: Library "stdune" not found.
# Hint: try: dune external-lib-deps --missing -p dune-build-info -j 2 @install
#   @runtest
# File "test/unit-tests/artifact_substitution/dune", line 3, characters 12-18:
# 3 |  (libraries stdune dune fiber dune._wp.dune dune_re)
#                 ^^^^^^

[avsm]

This dolmen.0.4 failure is a bit suspicious -- might be a regression in the menhir rules:

#=== ERROR while compiling dolmen.0.4 =========================================#
# context              2.0.4 | linux/x86_64 | ocaml-base-compiler.4.08.0 | git+file:///home/opam/opam-repository
# path                 ~/.opam/4.08/.opam-switch/build/dolmen.0.4
# command              ~/.opam/4.08/bin/dune build -p dolmen -j 1
# exit-code            1
# env-file             ~/.opam/log/dolmen-17-42ef32.env
# output-file          ~/.opam/log/dolmen-17-42ef32.out
### output ###
#       menhir src/languages/dimacs/parseDimacs__mock.ml.mock (exit 1)
# (cd _build/default && /home/opam/.opam/4.08/bin/menhir --explain --table --external-tokens Tokens_dimacs tokens_dimacs.mly src/languages/dimacs/parseDimacs.mly --base src/languages/dimacs/parseDimacs --infer-write-query src/languages/dimacs/parseDimacs__mock.ml.mock)
# Error: tokens_dimacs.mly: No such file or directory
#       menhir src/languages/icnf/parseiCNF__mock.ml.mock (exit 1)
# (cd _build/default && /home/opam/.opam/4.08/bin/menhir --explain --table --external-tokens Tokens_iCNF tokens_iCNF.mly src/languages/icnf/parseiCNF.mly --base src/languages/icnf/parseiCNF --infer-write-query src/languages/icnf/parseiCNF__mock.ml.mock)
# Error: tokens_iCNF.mly: No such file or directory
#       menhir src/languages/smtlib/parseSmtlib__mock.ml.mock (exit 1)
# (cd _build/default && /home/opam/.opam/4.08/bin/menhir --explain --table --external-tokens Tokens_smtlib tokens_smtlib.mly src/languages/smtlib/parseSmtlib.mly --base src/languages/smtlib/parseSmtlib --infer-write-query src/languages/smtlib/parseSmtlib__mock.ml.mock)
# Error: tokens_smtlib.mly: No such file or directory
#       menhir src/languages/tptp/parseTptp__mock.ml.mock (exit 1)
# (cd _build/default && /home/opam/.opam/4.08/bin/menhir --explain --table --external-tokens Tokens_tptp tokens_tptp.mly src/languages/tptp/parseTptp.mly --base src/languages/tptp/parseTptp --infer-write-query src/languages/tptp/parseTptp__mock.ml.mock)
# Error: tokens_tptp.mly: No such file or directory
#       menhir src/languages/zf/parseZf__mock.ml.mock (exit 1)
# (cd _build/default && /home/opam/.opam/4.08/bin/menhir --explain --table --external-tokens Tokens_zf tokens_zf.mly src/languages/zf/parseZf.mly --base src/languages/zf/parseZf --infer-write-query src/languages/zf/parseZf__mock.ml.mock)
# Error: tokens_zf.mly: No such file or directory
#     ocamlopt src/standard/.dolmen_std.objs/native/dolmen_std__Id.{cmx,o}

[rgrinberg]

This dolmen.0.4 failure is a bit suspicious -- might be a regression in the menhir rules:

I can confirm that this error occurs with dune 1.10.0 as well. The build-info thing is indeed a release blocker however

[avsm]

This msat.0.8 thing also a possible regression:

- conflict0 ... ok
- test_clean... ok
-     ocamlopt src/sudoku/sudoku_solve.exe (exit 2)
- (cd _build/default && /home/opam/.opam/4.08/bin/ocamlopt.opt -w -40 -warn-error -a -w +a-4-42-44-48-50-58-32-60@8 -color always -safe-string -g -O3 -bin-annot -unbox-closures -unbox-closures-factor 20 -o src/sudoku/sudoku_solve.exe -I /home/opam/.opam/4.08/lib/bytes -I /home/opam/.opam/4.08/lib/containers -I /home/opam/.opam/4.08/lib/containers/monomorphic -I /home/opam/.opam/4.08/lib/iter -I /home/opam/.opam/4.08/lib/result -I /home/opam/.opam/4.08/lib/sequence -I /home/opam/.opam/4.08/lib/uchar -I src/backtrack -I src/core /home/opam/.opam/4.08/lib/result/result.cmxa /home/opam/.opam/4.08/lib/iter/iter.cmxa src/core/msat.cmxa src/backtrack/msat_backtrack.cmxa /home/opam/.opam/4.08/lib/sequence/sequence.cmxa /home/opam/.opam/4.08/lib/sequence/sexpr.cmxa /home/opam/.opam/4.08/lib/containers/monomorphic/containers_monomorphic.cmxa /home/opam/.opam/4.08/lib/containers/containers.cmxa src/sudoku/.sudoku_solve.eobjs/native/sudoku_solve.cmx)
- gcc: error: /home/opam/.opam/4.08/lib/sequence/sexpr.a: No such file or directory
- File "caml_startup", line 1:
- Error: Error during linking
-         make alias tests/runtest
- test problem regression/regression1.icnf
- test problem regression/regression2.icnf
-         time alias tests/runtest
- run benchmarks…

[rgrinberg]

I'm unable to reproduce the msat issue with. @c-cube any ideas?

[c-cube]

I have no idea. I've seen that before but it's strange (happens in a test executable and I don't see any dependency to Sequence anywhere…)

[mseri]

The msat failure has been happening for a while, it was pointed out on some other non-dune PR not long ago