One Directory per Package #12614

rgrinberg · 2025-10-22T15:40:37Z

This PR introduces a (dir ..) field for packages in the dune-project file. The meaning of this field is to designate a single directory for a package.

We get two types of validation from this field:

Stanzas inside a package's directory may only belong to said package.
Stanzas outside the the package's directory may not be attached to said package.

Moreover, stanzas marked using dir field are excluded via --only-packages just like stanzas that use the package field.

Fixes #3255

rgrinberg · 2025-10-22T15:41:12Z

For now this is a draft as I need to fix some error messages and add locations to the errors.

rgrinberg · 2025-10-23T01:00:32Z

Errors and locations should be good now

test/blackbox-tests/test-cases/exclusive-package-dirs.t

src/dune_lang/package_id.mli

src/dune_lang/package_mask.mli

test/blackbox-tests/test-cases/exclusive-package-dirs.t

gridbugs

Generally looks good. I've requested some documentation here and there to help clarify some stuff not immediately apparent from the code. Don't forget to write a changelog entry.

Signed-off-by: Rudi Grinberg <me@rgrinberg.com>

In future work, we'd like to introduce a package to every single stanza, but doing that naively will create cycles. Signed-off-by: Rudi Grinberg <me@rgrinberg.com>

to break some cycles Signed-off-by: Rudi Grinberg <me@rgrinberg.com>

Signed-off-by: Rudi Grinberg <me@rgrinberg.com>

The dir stanza allows us to associate a directory tree with a particular package. This gives us two new facts: 1. If a stanza exists under this directory tree, it must belong to said package 2. If a stanza exists under another directory tree, it cannot belong to said package. Signed-off-by: Rudi Grinberg <me@rgrinberg.com>

@ElectreAAS

CHANGES: ### Fixed - Fix `include_subdirs qualified` incorrectly picking the furthest module instead of the closest when resolving module name ambiguities. (ocaml/dune#12587, @ElectreAAS and @Alizter) - Fix: include the module alias in the transitive dependency closure with `(include_subdirs qualified)`. (ocaml/dune#12299, @anmonteiro) - Improve error messages for invalid version formats containing non-ASCII characters. Previously, non-ASCII characters in version strings (e.g., `(lang dune è)` or `(using menhir π3.14)`) would fail with a generic "Invalid file" error. Now they display a clear message: "Invalid atom: contains non-ASCII character(s). Atoms must only contain ASCII characters." The fix is implemented at the lexer level, providing consistent error handling across all s-expression parsing. (ocaml/dune#12844, fixes ocaml/dune#12836, @benodiwal) - Pass private modules with -H when this is available (ocaml/dune#12666, @rgrinberg) - Allow multiple modules in `(modules_flags ...)`, in `coq.theory` (ocaml/dune#12733, @rlepigre) - Improve error message for invalid version formats in both `(lang dune ...)` and `(using extension ...)` declarations. Changes "Atom of the form NNN.NNN expected" to "Invalid version. Version must be two numbers separated by a dot." (ocaml/dune#12833, @benodiwal) - Fix crash when running `dune build @check` on a library with virtual modules. (ocaml/dune#12644, fixes ocaml/dune#12636, @Alizter) - Provide a more informative error message when `(pkg enabled)` is put in `dune-project` instead of `dune-workspace`. (ocaml/dune#12802, fixes ocaml/dune#12801, @benodiwal) - Improve error message when invalid version strings are used in `dune-project` files. Non-ASCII characters and malformed versions now show a helpful hint with an example of the correct format. (ocaml/dune#12794, fixes ocaml/dune#12751, @benodiwal) - Stop hiding the `root_module` from the include path (ocaml/dune#12239, @rgrinberg) - Allow `$ dune init` to work on absolute paths (ocaml/dune#12601, fixes ocaml/dune#7806, @rgrinberg) - `(include_subdirs qualified)`: Add missing alias dependency to module group. (ocaml/dune#12530, @anmonteiro) - Add Melange compilation to the `@all` alias in libraries (ocaml/dune#12628, @anmonteiro) - Fix greedy version location in lang declarations. Previously, error locations for invalid lang versions would span multiple bytes for multi-byte UTF-8 characters, causing carets to appear misaligned and seemingly include the closing parenthesis. Now, error locations for ASCII strings show the full length (e.g., "Ali" shows `^^^`), while non-ASCII strings show only the first byte (e.g., "è" shows `^`) to avoid multi-byte character display issues. (ocaml/dune#12869, fixes ocaml/dune#12806, @benodiwal) - melange support: don't emit empty JavaScript modules for generated module aliases. (ocaml/dune#12464, @anmonteiro) ### Added - (Experimental): Introduce the `library_parameter` stanza. It allows users to declare a parameter when using the OxCaml compiler. (ocaml/dune#11963, implements ocaml/dune#12084, @maiste) - Added the ability to scroll horizontally in TUI. (ocaml/dune#12386, @Alizter) - Feature: Include shell command that was executed when a cram test has occurred in the error message (ocaml/dune#12307, @rgrinberg) - support expanding variables in `(promote (into ..))` (ocaml/dune#12832, fixes ocaml/dune#12742, @anmonteiro) - Add support for `%{cmt:...}` and `%{cmti:...}` variables to reference compiled annotation files (.cmt and .cmti) containing typed abstract syntax trees with location and type information. (ocaml/dune#12634, grants ocaml/dune#12633, @Alizter) - Add `$ dune describe tests` to describe the tests in the workspace (@Gromototo, ocaml/dune#12545, fixes ocaml/dune#12030) - Add `argv`, the process environment, and the dune version to the config event in the trace (ocaml/dune#12909, @rgrinberg) - Allow `dune runtest` to properly run while a watch mode server is running. (ocaml/dune#12473, grants ocaml/dune#8114, @gridbugs and @ElectreAAS) - Use copy-on-write (COW) when copying files on filesystems that support it (Btrfs, ZFS, XFS, etc), under Linux. (ocaml/dune#12074, fixes ocaml/dune#12071, @nojb) - Add support for Tangled ATproto-based code repositories (ocaml/dune#12197, @avsm) - Add support for instantiating OxCaml parameterised libraries. (ocaml/dune#12561, @art-w) - Add a `(conflict_markers error|ignore)` option to the cram stanza. When `(conflict_markers error)` is set, the cram test will fail in the presence of conflict markers. Git, diff3 and jujutsu conflict markers are detected. (ocaml/dune#12538, ocaml/dune#12617, ocaml/dune#12655, fixes ocaml/dune#12512, @rgrinberg, @Alizter) - Introduce a `%{ppx:lib1+..+libn}` stanza to make it possible to refer to ppx executables built by dune. This is useful for writing tests (ocaml/dune#12711, @rgrinberg) - Introduce a `(dir ..)` field on packages defined in the `dune-project`. This field allows to associate a directory with a particular package. This makes dune automatically filter out all stanzas in this directory and its descendants with `--only-packages`. All users are recommended to switch to using this field. (ocaml/dune#12614, fixes ocaml/dune#3255, @rgrinberg) - Add support for `DUNE_ROOT` environment variable, similar to the existing `--root` CLI parameter. (fixes ocaml/dune#12399 @sir4ur0n) - Introduce an `unused-libs` alias to detect unused libraries. (ocaml/dune#12623, fixes ocaml/dune#650, @rgrinberg) - Add `--files` flag to `dune describe opam-files` to print only the names of the opam files line by line. (ocaml/dune#9793, @reynir and @Alizter) - `dune exec` now accepts absolute paths inside the workspace. (ocaml/dune#12094, @Alizter) - Add `coqdoc_header` and `coqdoc_footer` fields to the `coq` field of the `env` stanza, and to the `coq.theory` stanza, allowing to configure a custom header or footer respectively in the HTML output of `coqdoc`. (ocaml/dune#11131, @rlepigre) - Allow `dune fmt` to properly run while a watch mode server is running. Note that the `--preview` flag is not supported in this mode. (ocaml/dune#12064, @ElectreAAS) - Support for generating `_CoqProject` files for `coq.theory` stanzas. (ocaml/dune#11752, @rlepigre) - Added `(files)` stanza, similar to `(dirs)` to control which files are visible to Dune on a per-directory basis. (ocaml/dune#12879, @nojb) - Add support for %{ocaml-config:ox} (ocaml/dune#12236, @jonludlam) - Introduce `dune promotion show` command to display the contents of corrected files that are ready for promotion. This allows users to preview changes before running `dune promote`. The command accepts file arguments to show specific files, or displays all promotable files when called without arguments. (ocaml/dune#12669, fixes ocaml/dune#3883, @MixiMaxiMouse) - New `(lang rocq)` build mode for Rocq 9.0 and later. This new mode is very similar to the existing `(lang coq)`, except that it doesn't need the `coq*` compatibility wrappers. As of today `(lang rocq)` doesn't support yet composed builds with Rocq itself, this will be added later. `(lang coq)` is deprecated, development is frozen, and will be removed at some point in the future. (ocaml/dune#12035, @ejgallego, @Lysxia, fixes ocaml/dune#11572) ### Changed - Don't run `ocamldep` to compute false dependencies on the `root_module` (ocaml/dune#12227, @rgrinberg) - `dune format-dune-file` now uses the syntax version of the Dune project that contains the file being formatted (if any) instead of using the latest version available, which remains the default if there is no Dune project in scope. (ocaml/dune#11865, @nojb) - Persistent DB and process events have been slightly modified. Persistent DB events have more concise names and job events always include full information. (ocaml/dune#12867, @rgrinberg) - Removed the `--trace-extended` flag. Its functionality is always enabled when tracing is active (ocaml/dune#12908, @rgrinberg) - The `test/dune` file generated by `dune init proj` now depends on the project library. (ocaml/dune#12791, @shonfeder) - Starting with version 3.21 of the Dune language, Dune no longer changes the default set of compiler warnings. For users that would like to keep the old behaviour, the variable `%{dune-warnings}` can be used in an `(env)` stanza in a top-level Dune file: `(env (dev (flags :standard %{dune-warnings})))`. (ocaml/dune#12766, @nojb) - Fix: stop generating `cmt` files for cinaps binaries (ocaml/dune#12530, @rgrinberg)

@ElectreAAS

CHANGES: ### Fixed - Fix `include_subdirs qualified` incorrectly picking the furthest module instead of the closest when resolving module name ambiguities. (ocaml/dune#12587, @ElectreAAS and @Alizter) - Fix: include the module alias in the transitive dependency closure with `(include_subdirs qualified)`. (ocaml/dune#12299, @anmonteiro) - Improve error messages for invalid version formats containing non-ASCII characters. Previously, non-ASCII characters in version strings (e.g., `(lang dune è)` or `(using menhir π3.14)`) would fail with a generic "Invalid file" error. Now they display a clear message: "Invalid atom: contains non-ASCII character(s). Atoms must only contain ASCII characters." The fix is implemented at the lexer level, providing consistent error handling across all s-expression parsing. (ocaml/dune#12844, fixes ocaml/dune#12836, @benodiwal) - Pass private modules with -H when this is available (ocaml/dune#12666, @rgrinberg) - Allow multiple modules in `(modules_flags ...)`, in `coq.theory` (ocaml/dune#12733, @rlepigre) - Improve error message for invalid version formats in both `(lang dune ...)` and `(using extension ...)` declarations. Changes "Atom of the form NNN.NNN expected" to "Invalid version. Version must be two numbers separated by a dot." (ocaml/dune#12833, @benodiwal) - Fix crash when running `dune build @check` on a library with virtual modules. (ocaml/dune#12644, fixes ocaml/dune#12636, @Alizter) - Provide a more informative error message when `(pkg enabled)` is put in `dune-project` instead of `dune-workspace`. (ocaml/dune#12802, fixes ocaml/dune#12801, @benodiwal) - Improve error message when invalid version strings are used in `dune-project` files. Non-ASCII characters and malformed versions now show a helpful hint with an example of the correct format. (ocaml/dune#12794, fixes ocaml/dune#12751, @benodiwal) - Stop hiding the `root_module` from the include path (ocaml/dune#12239, @rgrinberg) - Allow `$ dune init` to work on absolute paths (ocaml/dune#12601, fixes ocaml/dune#7806, @rgrinberg) - `(include_subdirs qualified)`: Add missing alias dependency to module group. (ocaml/dune#12530, @anmonteiro) - Add Melange compilation to the `@all` alias in libraries (ocaml/dune#12628, @anmonteiro) - Fix greedy version location in lang declarations. Previously, error locations for invalid lang versions would span multiple bytes for multi-byte UTF-8 characters, causing carets to appear misaligned and seemingly include the closing parenthesis. Now, error locations for ASCII strings show the full length (e.g., "Ali" shows `^^^`), while non-ASCII strings show only the first byte (e.g., "è" shows `^`) to avoid multi-byte character display issues. (ocaml/dune#12869, fixes ocaml/dune#12806, @benodiwal) - melange support: don't emit empty JavaScript modules for generated module aliases. (ocaml/dune#12464, @anmonteiro) ### Added - (Experimental): Introduce the `library_parameter` stanza. It allows users to declare a parameter when using the OxCaml compiler. (ocaml/dune#11963, implements ocaml/dune#12084, @maiste) - Added the ability to scroll horizontally in TUI. (ocaml/dune#12386, @Alizter) - Feature: Include shell command that was executed when a cram test has occurred in the error message (ocaml/dune#12307, @rgrinberg) - support expanding variables in `(promote (into ..))` (ocaml/dune#12832, fixes ocaml/dune#12742, @anmonteiro) - Add support for `%{cmt:...}` and `%{cmti:...}` variables to reference compiled annotation files (.cmt and .cmti) containing typed abstract syntax trees with location and type information. (ocaml/dune#12634, grants ocaml/dune#12633, @Alizter) - Add `$ dune describe tests` to describe the tests in the workspace (@Gromototo, ocaml/dune#12545, fixes ocaml/dune#12030) - Add `argv`, the process environment, and the dune version to the config event in the trace (ocaml/dune#12909, @rgrinberg) - Allow `dune runtest` to properly run while a watch mode server is running. (ocaml/dune#12473, grants ocaml/dune#8114, @gridbugs and @ElectreAAS) - Use copy-on-write (COW) when copying files on filesystems that support it (Btrfs, ZFS, XFS, etc), under Linux. (ocaml/dune#12074, fixes ocaml/dune#12071, @nojb) - Add support for Tangled ATproto-based code repositories (ocaml/dune#12197, @avsm) - Add support for instantiating OxCaml parameterised libraries. (ocaml/dune#12561, @art-w) - Add a `(conflict_markers error|ignore)` option to the cram stanza. When `(conflict_markers error)` is set, the cram test will fail in the presence of conflict markers. Git, diff3 and jujutsu conflict markers are detected. (ocaml/dune#12538, ocaml/dune#12617, ocaml/dune#12655, fixes ocaml/dune#12512, @rgrinberg, @Alizter) - Introduce a `%{ppx:lib1+..+libn}` stanza to make it possible to refer to ppx executables built by dune. This is useful for writing tests (ocaml/dune#12711, @rgrinberg) - Introduce a `(dir ..)` field on packages defined in the `dune-project`. This field allows to associate a directory with a particular package. This makes dune automatically filter out all stanzas in this directory and its descendants with `--only-packages`. All users are recommended to switch to using this field. (ocaml/dune#12614, fixes ocaml/dune#3255, @rgrinberg) - Add support for `DUNE_ROOT` environment variable, similar to the existing `--root` CLI parameter. (fixes ocaml/dune#12399 @sir4ur0n) - Introduce an `unused-libs` alias to detect unused libraries. (ocaml/dune#12623, fixes ocaml/dune#650, @rgrinberg) - Add `--files` flag to `dune describe opam-files` to print only the names of the opam files line by line. (ocaml/dune#9793, @reynir and @Alizter) - `dune exec` now accepts absolute paths inside the workspace. (ocaml/dune#12094, @Alizter) - Add `coqdoc_header` and `coqdoc_footer` fields to the `coq` field of the `env` stanza, and to the `coq.theory` stanza, allowing to configure a custom header or footer respectively in the HTML output of `coqdoc`. (ocaml/dune#11131, @rlepigre) - Allow `dune fmt` to properly run while a watch mode server is running. Note that the `--preview` flag is not supported in this mode. (ocaml/dune#12064, @ElectreAAS) - Support for generating `_CoqProject` files for `coq.theory` stanzas. (ocaml/dune#11752, @rlepigre) - Added `(files)` stanza, similar to `(dirs)` to control which files are visible to Dune on a per-directory basis. (ocaml/dune#12879, @nojb) - Add support for %{ocaml-config:ox} (ocaml/dune#12236, @jonludlam) - Introduce `dune promotion show` command to display the contents of corrected files that are ready for promotion. This allows users to preview changes before running `dune promote`. The command accepts file arguments to show specific files, or displays all promotable files when called without arguments. (ocaml/dune#12669, fixes ocaml/dune#3883, @MixiMaxiMouse) - New `(lang rocq)` build mode for Rocq 9.0 and later. This new mode is very similar to the existing `(lang coq)`, except that it doesn't need the `coq*` compatibility wrappers. As of today `(lang rocq)` doesn't support yet composed builds with Rocq itself, this will be added later. `(lang coq)` is deprecated, development is frozen, and will be removed at some point in the future. (ocaml/dune#12035, @ejgallego, @Lysxia, fixes ocaml/dune#11572) ### Changed - Don't run `ocamldep` to compute false dependencies on the `root_module` (ocaml/dune#12227, @rgrinberg) - `dune format-dune-file` now uses the syntax version of the Dune project that contains the file being formatted (if any) instead of using the latest version available, which remains the default if there is no Dune project in scope. (ocaml/dune#11865, @nojb) - Persistent DB and process events have been slightly modified. Persistent DB events have more concise names and job events always include full information. (ocaml/dune#12867, @rgrinberg) - Removed the `--trace-extended` flag. Its functionality is always enabled when tracing is active (ocaml/dune#12908, @rgrinberg) - The `test/dune` file generated by `dune init proj` now depends on the project library. (ocaml/dune#12791, @shonfeder) - Starting with version 3.21 of the Dune language, Dune no longer changes the default set of compiler warnings. For users that would like to keep the old behaviour, the variable `%{dune-warnings}` can be used in an `(env)` stanza in a top-level Dune file: `(env (dev (flags :standard %{dune-warnings})))`. (ocaml/dune#12766, @nojb) - Fix: stop generating `cmt` files for cinaps binaries (ocaml/dune#12530, @rgrinberg)

rgrinberg requested a review from gridbugs October 22, 2025 15:40

rgrinberg force-pushed the exclusive-package-dir branch 3 times, most recently from 61f91c0 to 325e38e Compare October 23, 2025 01:00

rgrinberg marked this pull request as ready for review October 23, 2025 01:00

rgrinberg force-pushed the exclusive-package-dir branch 2 times, most recently from 600a78a to d2ef7fa Compare October 23, 2025 17:10