Exception "Build_system.get_collector called on closed directory"

[Kakadu]

➜  1 jbuilder build
Internal error, please report upstream including the contents of _build/log.
Description: ("Build_system.get_collector called on closed directory"
 (dir _build/default/lib/stubs))
Backtrace:
Raised at file "src/errors.ml", line 17, characters 25-45
Called from file "src/build_system.ml", line 1468, characters 18-47
Called from file "src/gen_rules.ml", line 519, characters 4-570
Called from file "list.ml", line 82, characters 20-23
Called from file "src/gen_rules.ml", line 642, characters 12-100
Called from file "src/stdune/exn.ml", line 8, characters 8-11
Re-raised at file "src/stdune/exn.ml", line 10, characters 30-37
Called from file "src/gen_rules.ml", line 958, characters 13-72
Called from file "src/stdune/list.ml", line 13, characters 10-13
Called from file "src/gen_rules.ml", line 954, characters 4-773
Called from file "src/gen_rules.ml", line 993, characters 19-30
Called from file "src/build_system.ml", line 860, characters 6-62
Called from file "src/build_system.ml", line 836, characters 6-59
Re-raised at file "src/build_system.ml", line 847, characters 6-17
Called from file "src/build_system.ml", line 804, characters 32-63
Called from file "src/build_system.ml", line 814, characters 4-24
Called from file "src/build_interpret.ml", line 101, characters 24-40
Called from file "src/build_interpret.ml", line 60, characters 31-43
Called from file "src/build_interpret.ml", line 60, characters 31-43
Called from file "src/build_system.ml", line 1184, characters 10-108
Called from file "src/fiber/fiber.ml", line 359, characters 6-13
➜  1 cat lib/jbuild 
(jbuild_version 1)
(library
 ((name            lablqml)
  (modules         (lablqml))
  (cxx_names         (stubs/CamlPropertyMap_stubs  
                    stubs/object_ml
                    stubs/QSingleFunc_stubs
                    stubs/stubs
                    stubs/variant
                    stubs/modelindex
                    stubs/propMap_stubs
                    stubs/startup_stubs
                    stubs/test_stubs
    ) )
  ; dirty hack next line
  (cxx_flags         ( (:include c_flags.sexp) -O3 -I ../../stubs -Dprivate=public))
  (c_library_flags (:include c_library_flags.sexp) )
))

(rule
 ((targets (c_flags.sexp
            c_library_flags.sexp))
  (deps    (config/discover.exe))
  (action  (run ${<} -ocamlc ${OCAMLC}))))

➜  1 find .
.
./lib
./lib/.merlin
./lib/src_tests
./lib/src_tests/.merlin
./lib/src_tests/test1.qml
./lib/src_tests/test3.ml
./lib/src_tests/test2.qml
./lib/src_tests/test2.ml
./lib/src_tests/test4.qml
./lib/src_tests/test4.ml
./lib/src_tests/test1.ml
./lib/src_tests/test3.qml
./lib/README.md
./lib/liblablqml_stubs.clib
./lib/jbuild
./lib/myocamlbuild.ml
./lib/_oasis
./lib/config
./lib/config/.merlin
./lib/config/discover.ml
./lib/config/jbuild
./lib/lablqml.ml
./lib/lablqml.mllib
./lib/Makefile.jbuilder
./lib/Makefile
./lib/META
./lib/stubs
./lib/stubs/lablqml.h
./lib/stubs/QSingleFunc_stubs.cpp
./lib/stubs/CamlPropertyMap.h
./lib/stubs/QSingleFunc.h
./lib/stubs/propMap_stubs.cpp
./lib/stubs/variant.cpp
./lib/stubs/object_ml.cpp
./lib/stubs/startup_stubs.cpp
./lib/stubs/modelindex.cpp
./lib/stubs/stubs.cpp
./lib/stubs/CamlPropertyMap_stubs.cpp
./lib/stubs/object.h
./lib/stubs/test_stubs.cpp
./lib/stubs/variant.h
./lib/stubs/_build
./lib/stubs/_build/.db
./lib/stubs/_build/.digest-db
./lib/stubs/_build/.to-delete-in-source-tree
./lib/stubs/_build/log
./lib/lablqml.mli
./lib/_tags
./lib/.gitignore
./_build
./_build/.db
./_build/.universe-state
./_build/.digest-db
./_build/.to-delete-in-source-tree
./_build/log
➜  1 
➜  1 cat _build/log 
# jbuilder build
# OCAMLPARAM: unset
$ /home/kakadu/.opam/4.06.1+fp+flambda/bin/ocamlc.opt -config > /tmp/jbuild2cf16b.output

[ghost]

The error is bad and needs to be fixed. The problem in this case is that dune expects all the files attached to a library to be in the same directory. What you need to do instead of writing stubs/CamlPropertyMap_stubs is tell dune to copy over the .cpp files in the current directory with:

(copy_files# stubs/*.cpp)

We could eventually changes things so that this is not needed, but this requires a bit of work.

[rjbou]

This error reappears in dune 1.6.0, with ocaml-mccs, to reproduce, just install it opam install mccs -vv.
More logs, in ocaml-mccs' master:

$ jbuilder build -p mccs
Internal error, please report upstream including the contents of _build/log.
Description:
("Build_system.get_collector called on closed directory"
 (dir (In_build_dir default/src/glpk))
 (load_dir_stack ()))
Backtrace:
Raised at file "src/stdune/exn.ml", line 32, characters 5-10
Called from file "src/build_system.ml", line 1589, characters 18-47
Called from file "set.ml", line 343, characters 35-38
Called from file "src/stdune/set.ml" (inlined), line 36, characters 18-27
Called from file "src/install_rules.ml", line 71, characters 4-63
Called from file "map.ml", line 295, characters 20-25
Called from file "src/gen_rules.ml", line 239, characters 4-25
Called from file "map.ml", line 295, characters 20-25
Called from file "src/gen_rules.ml", line 313, characters 2-60
Called from file "src/fiber/fiber.ml", line 111, characters 22-27
Called from file "src/fiber/fiber.ml", line 299, characters 6-13

I must not segfault.  Uncertainty is the mind-killer.  Exceptions are
the little-death that brings total obliteration.  I will fully express
my cases.  Execution will pass over me and through me.  And when it
has gone past, I will unwind the stack along its path.  Where the
cases are handled there will be nothing.  Only I will remain.

$ cat _build/log
# jbuilder build -p mccs
# OCAMLPARAM: unset
$ /usr/bin/nproc > /tmp/dune214718.output 2> /dev/null
$ [...]ocamlc.opt -config > /tmp/dune95c50b.output

[hongchangwu]

I can confirm this as well. Can we re-open this one? Seems like a regression.

[ghost]

Sure

[ghost]

I had a look and it seems that it is gen_lib_dune_file that is not called at the right time. We should probably move that to lib_rules.ml. /cc @rgrinberg

[rgrinberg]

If we move it to lib_rules.ml, we'll need to add on_load_dir for the install directory I think. I still don't quite understand, why wasn't this an issue before? gen_lib_dune_file doesn't seem to be forcing the evaluation of any rules.

Nvm, I realized that we generate the installed dune file in the lib's source directory.

[rgrinberg]

fixed in #1612

[rjbou]

Thanks!