Mdx stanza doesn't work with multiple packages using opam

[Risto-Stevcev]

Expected Behavior

Trying opam install . -t -d should install as expected with the mdx stanza. The stanza works fine when developing locally with runtest.

Actual Behavior

It errors out with this when trying to build the first package (my_package):

File "dune", line 1, characters 0-91:
1 | (mdx
2 |  (files README.md)
3 |  (packages my_project my_project_async my_project_lwt))
Error: No rule found for alias .my_project_async-files
File "dune", line 1, characters 0-91:
1 | (mdx
2 |  (files README.md)
3 |  (packages my_project my_project_async my_project_lwt))
Error: No rule found for alias .my_project_lwt-files

Reproduction

Directory structure:

.
├ dune-project
├ dune
├ README.md
├ my_project/
   └ src/
      ├ dune
      └ Foo.ml
├ my_project_lwt/
   └ src/
      ├ dune
      └ Bar.ml
└ my_project_async/
   └ src/
      ├ dune
      └ Baz.ml

./dune-project:

(lang dune 2.4.0)
(generate_opam_files true)
(implicit_transitive_deps false)
(using mdx 0.1)
...

(package
  (name my_project)
  ...)
(package
  (name my_project_lwt)
  ...)
(package
  (name my_project_async)
  ...)

./dune:

(mdx
 (files README.md)
 (packages my_project my_project_async my_project_lwt))

./my_project/src/dune

(library
 (name my_project)
  ...)

./my_project_lwt/src/dune

(library
 (name my_project_lwt)
  ...)

./my_project_async/src/dune

(library
 (name my_project_async)
  ...)

Specifications

$ dune --version
2.4.0
$ ocamlc --version
4.10.0

I'm using Arch Linux:

$ uname -a
Linux ristoarch 5.4.13-arch1-1 #1 SMP PREEMPT Fri, 17 Jan 2020 23:09:54 +0000 x86_64 GNU/Linux

[ghost]

That doesn't seem easy to fix. TBH, it's quite difficult to get the tests to work well when building packages by packages in isolation. If that's for the CI, the current trend is to run the testsuite with dune directly in the CI.

[Risto-Stevcev]

Is there a workaround for it? Right now I'm doing this so that it can compile with opam:

; dune
(rule
 (alias runtest)
 (package my_project)
 (deps
  (package my_project)
  (:x README.md))
 (action
  (progn
   (run ocaml-mdx test %{x})
   (diff? %{x} %{x}.corrected))))

The rule.package field makes sure that it only compiles for the my_project package and not the my_project_lwt or my_project_async versions, but the downside is that I can't reference the lwt and async stuff in the README. I can write separate readmes for those packages but ideally I'd like to be able to reference it all in one readme.

[ghost]

Regarding the package field to attach the rule to a particular package, we are working on a more general solution allowing to tag a directory as belonging to a particular package. Then, during an opam build directories from other packages will be completely ignored. This will simplify things a lot.

Regarding the original issue, unfortunately there is no workaround at the moment. Why do you use the -t option of opam install BTW?

[Risto-Stevcev]

I'm new to the opam workflow. I'm trying to get lambda_streams published but opam publish was failing for me with a git fetch error.

I tried doing it manually by forking the repo and submitting a PR, but the travis build was failing. I decided to test the repo locally by doing opam install . -t -d to make sure everything works to avoid spending too much time on the PR. I'm not sure if opam-repositorys travis build checks for that though, or if opam install . would've been enough to test locally before trying to publish.

[ghost]

I see. Are you using dune to generate the opam file?

[Risto-Stevcev]

Yeah

[ghost]

Ok, so one thing you can do is create a <package>.opam.template file, copy the build: field from the generated <package>.opam file to the template file, and remove the "runtest" {with-test} bit. That will prevent opam install -t from running the test.

I'm thinking that we should add an option in the dune-project file to skip the runtest part when generating the opam files. Other users have been bitten by the same issue.

[emillon]

Hi,

The new version of the (mdx) stanza allows attaching a stanza to particular package. So you can express your workaround more simply as

; dune
(mdx
 (files README.md); or just remove this if this is the only md file
 (package my_project))

As for the other question - how to deal with tests that are not specific to a single opam package - a common pattern is to create a dedicated "mypackage-test" package and attach these tests to it. You can find an example in https://github.com/mirage/irmin with the irmin-test package.

Please reopen or file a new issue if that's not enough to support your use case. Thanks!