Linking with core_bench fails in a bytecode-only switch

[fpottier]

Hello,

I am unable to build a program that uses core_bench under OCaml 4.09.0+bytecode-only.

To reproduce the problem, the following instructions suffice:

git clone git@github.com:janestreet/core_bench.git
cd core_bench
git checkout v0.13.0
dune build @all

This produces the following output:

File "_none_", line 1:  
Error: Error while linking /home/fpottier/.opam/4.09.0+bytecode-only/lib/core/core.cma(Core__Linux_ext):
       The external function `core_linux_timerfd_settime' is not available
      ocamlc test/test_bench.exe (exit 2)
(cd _build/default && /home/fpottier/.opam/4.09.0+bytecode-only/bin/ocamlc -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -o test/test_bench.exe -custom /home/fpottier/.opam/4.09.0+bytecode-only/lib/base/base_internalhash_types/base_internalhash_types.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/base/base_internalhash_types /home/fpottier/.opam/4.09.0+bytecode-only/lib/base/caml/caml.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/sexplib0/sexplib0.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/base/shadow_stdlib/shadow_stdlib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/base/base.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/base /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_compare/runtime-lib/ppx_compare_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_sexp_conv/runtime-lib/ppx_sexp_conv_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_hash/runtime-lib/ppx_hash_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_enumerate/runtime-lib/ppx_enumerate_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/fieldslib/fieldslib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_here/runtime-lib/ppx_here_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_assert/runtime-lib/ppx_assert_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_bench/runtime-lib/ppx_bench_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_inline_test/config/inline_test_config.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_inline_test/runtime-lib/ppx_inline_test_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/splittable_random/splittable_random.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/base_quickcheck/base_quickcheck.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/base/md5/md5_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ocaml/unix.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/ocaml /home/fpottier/.opam/4.09.0+bytecode-only/lib/ocaml/bigarray.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/ocaml /home/fpottier/.opam/4.09.0+bytecode-only/lib/variantslib/variantslib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/bin_prot/shape/bin_shape_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/bin_prot/bin_prot.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/bin_prot /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_expect/config/expect_test_config.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_expect/common/expect_test_common.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_expect/collector/expect_test_collector.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_expect/collector /home/fpottier/.opam/4.09.0+bytecode-only/lib/typerep/typerep_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/stdio/stdio.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/jane-street-headers/jane_street_headers.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/time_now/time_now.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/time_now /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_module_timer/runtime/ppx_module_timer_runtime.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/base_bigstring/base_bigstring.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/base_bigstring /home/fpottier/.opam/4.09.0+bytecode-only/lib/core_kernel/base_for_tests/base_for_tests.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/parsexp/parsexp.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/sexplib/sexplib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/core_kernel/core_kernel.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/core_kernel /home/fpottier/.opam/4.09.0+bytecode-only/lib/ocaml/threads/threads.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/ocaml /home/fpottier/.opam/4.09.0+bytecode-only/lib/core/error_checking_mutex/error_checking_mutex.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/core/error_checking_mutex /home/fpottier/.opam/4.09.0+bytecode-only/lib/core_kernel/bounded_int_table/bounded_int_table.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/core_kernel/flags/flags.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/core_kernel/version_util/version_util.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/core_kernel/version_util /home/fpottier/.opam/4.09.0+bytecode-only/lib/sexplib/unix/sexplib_unix.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/spawn/spawn.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/spawn /home/fpottier/.opam/4.09.0+bytecode-only/lib/core/core.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/core /home/fpottier/.opam/4.09.0+bytecode-only/lib/textutils/console/console.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/textutils/ascii_table/ascii_table.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/core/time_stamp_counter/time_stamp_counter.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/core/time_stamp_counter src/core_bench.cma test/.test_bench.eobjs/byte/allocation_tests.cmo test/.test_bench.eobjs/byte/array_tests.cmo test/.test_bench.eobjs/byte/basic_tests.cmo test/.test_bench.eobjs/byte/exception_tests.cmo test/.test_bench.eobjs/byte/gc_tests.cmo test/.test_bench.eobjs/byte/tsc.cmo test/.test_bench.eobjs/byte/tuple_tests.cmo test/.test_bench.eobjs/byte/test_bench.cmo)
/tmp/ccKRPh7s.o:(.data+0x2b78): undefined reference to `core_linux_timerfd_settime'
/tmp/ccKRPh7s.o:(.data+0x2b80): undefined reference to `core_linux_timerfd_gettime'
/tmp/ccKRPh7s.o:(.data+0x2b88): undefined reference to `core_linux_timerfd_create'
/tmp/ccKRPh7s.o:(.data+0x2bc0): undefined reference to `core_linux_timerfd_CLOCK_REALTIME'
/tmp/ccKRPh7s.o:(.data+0x2bc8): undefined reference to `core_linux_timerfd_CLOCK_MONOTONIC'
/tmp/ccKRPh7s.o:(.data+0x2bd0): undefined reference to `core_linux_timerfd_TFD_NONBLOCK'
/tmp/ccKRPh7s.o:(.data+0x2bd8): undefined reference to `core_linux_timerfd_TFD_CLOEXEC'
collect2: error: ld returned 1 exit status
File "_none_", line 1:
Error: Error while building custom runtime system
      ocamlc test/mv_regr.exe (exit 2)
(cd _build/default && /home/fpottier/.opam/4.09.0+bytecode-only/bin/ocamlc -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -o test/mv_regr.exe -custom /home/fpottier/.opam/4.09.0+bytecode-only/lib/base/base_internalhash_types/base_internalhash_types.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/base/base_internalhash_types /home/fpottier/.opam/4.09.0+bytecode-only/lib/base/caml/caml.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/sexplib0/sexplib0.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/base/shadow_stdlib/shadow_stdlib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/base/base.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/base /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_compare/runtime-lib/ppx_compare_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_sexp_conv/runtime-lib/ppx_sexp_conv_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_hash/runtime-lib/ppx_hash_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_enumerate/runtime-lib/ppx_enumerate_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/fieldslib/fieldslib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_here/runtime-lib/ppx_here_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_assert/runtime-lib/ppx_assert_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_bench/runtime-lib/ppx_bench_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_inline_test/config/inline_test_config.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_inline_test/runtime-lib/ppx_inline_test_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/splittable_random/splittable_random.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/base_quickcheck/base_quickcheck.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/base/md5/md5_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ocaml/unix.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/ocaml /home/fpottier/.opam/4.09.0+bytecode-only/lib/ocaml/bigarray.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/ocaml /home/fpottier/.opam/4.09.0+bytecode-only/lib/variantslib/variantslib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/bin_prot/shape/bin_shape_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/bin_prot/bin_prot.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/bin_prot /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_expect/config/expect_test_config.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_expect/common/expect_test_common.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_expect/collector/expect_test_collector.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_expect/collector /home/fpottier/.opam/4.09.0+bytecode-only/lib/typerep/typerep_lib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/stdio/stdio.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/jane-street-headers/jane_street_headers.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/time_now/time_now.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/time_now /home/fpottier/.opam/4.09.0+bytecode-only/lib/ppx_module_timer/runtime/ppx_module_timer_runtime.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/base_bigstring/base_bigstring.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/base_bigstring /home/fpottier/.opam/4.09.0+bytecode-only/lib/core_kernel/base_for_tests/base_for_tests.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/parsexp/parsexp.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/sexplib/sexplib.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/core_kernel/core_kernel.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/core_kernel /home/fpottier/.opam/4.09.0+bytecode-only/lib/ocaml/threads/threads.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/ocaml /home/fpottier/.opam/4.09.0+bytecode-only/lib/core/error_checking_mutex/error_checking_mutex.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/core/error_checking_mutex /home/fpottier/.opam/4.09.0+bytecode-only/lib/core_kernel/bounded_int_table/bounded_int_table.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/core_kernel/flags/flags.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/core_kernel/version_util/version_util.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/core_kernel/version_util /home/fpottier/.opam/4.09.0+bytecode-only/lib/sexplib/unix/sexplib_unix.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/spawn/spawn.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/spawn /home/fpottier/.opam/4.09.0+bytecode-only/lib/core/core.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/core /home/fpottier/.opam/4.09.0+bytecode-only/lib/textutils/console/console.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/textutils/ascii_table/ascii_table.cma /home/fpottier/.opam/4.09.0+bytecode-only/lib/core/time_stamp_counter/time_stamp_counter.cma -I /home/fpottier/.opam/4.09.0+bytecode-only/lib/core/time_stamp_counter src/core_bench.cma test/.test_bench.eobjs/byte/mv_regr.cmo)
/tmp/ccfrpcyq.o:(.data+0x2b78): undefined reference to `core_linux_timerfd_settime'
/tmp/ccfrpcyq.o:(.data+0x2b80): undefined reference to `core_linux_timerfd_gettime'
/tmp/ccfrpcyq.o:(.data+0x2b88): undefined reference to `core_linux_timerfd_create'
/tmp/ccfrpcyq.o:(.data+0x2bc0): undefined reference to `core_linux_timerfd_CLOCK_REALTIME'
/tmp/ccfrpcyq.o:(.data+0x2bc8): undefined reference to `core_linux_timerfd_CLOCK_MONOTONIC'
/tmp/ccfrpcyq.o:(.data+0x2bd0): undefined reference to `core_linux_timerfd_TFD_NONBLOCK'
/tmp/ccfrpcyq.o:(.data+0x2bd8): undefined reference to `core_linux_timerfd_TFD_CLOEXEC'
collect2: error: ld returned 1 exit status
File "_none_", line 1:
Error: Error while building custom runtime system

I am running Ubuntu LTS 16.04, in case that matters.

[xclerc]

If I remember correctly, some primitive definitions are indeed guarded
by #ifdefs; could you try to run the ./config/discover.sh script
(from the core repository), and post the generated contents?

[fpottier]

Okay. (By the way, discover.sh says run ./configure first, but the script is named detect.sh, not configure.) Using a freshly cloned core repository, I get this:

master $ more setup.data 
linux_possible="true"
posix_timers_possible="true"
rt_possible="true"
master $ cat setup.data 
linux_possible="true"
posix_timers_possible="true"
rt_possible="true"
master $ cat discover.out 
#ifndef CORE_DISCOVER_OUT
#define CORE_DISCOVER_OUT
#undef JSC_LINUX_EXT
#undef JSC_THREAD_ID
#undef JSC_POSIX_TIMERS
#define JSC_RLIMIT_NICE
#define JSC_RLIMIT_AS
#define JSC_ARCH_SIXTYFOUR
#define JSC_MSG_NOSIGNAL
#undef JSC_SO_NOSIGPIPE
#define JSC_FDATASYNC
#define JSC_THREAD_CPUTIME
#undef JSC_ARCH_x86_64
#undef JSC_ARCH_i386
#define JSC_WORDEXP
#undef JSC_RECVMMSG
#undef JSC_TIMERFD
#undef JSC_EVENTFD
#endif

[xclerc]

(Thanks for reporting the filename mismatch, it will be fixed
by the next push.)

The fact that JSC_TIMERFD is unset is probably the culprit;
the script unset it because the build of test_timerfd.c failed.
Presumably because "sys/timerfd.h" could not be included.

The following command should expose the underlying issue:

`ocamlc -config | grep bytecomp_c_compiler | cut -d ':' -f 2` config/test_timerfd.c

(to be run from the root of your copy of the core repository)

[fpottier]

Hi @xclerc. The command that you suggested succeeds (exit code 0, no output) and creates a.out. Running a.out also succeeds (exit code 0, no output). The file /usr/include/x86_64-linux-gnu/sys/timerfd.h exists on my system.

[xclerc]

Sorry, the ./discover.sh bit above was misleading, as it turns
out it relies on the script being called from dune, which will set
the bytecomp_c_compiler environment variable. I was not
able to reproduce the problem with the latest ubuntu, but will
now try with 16.04.

[xclerc]

I was unable to reproduce the problem with 16.04.

[fpottier]

What seems weird is that everything works fine in 4.09, but fails in 4.09+bytecode-only. So the problem is perhaps not related to the particular OS I am using?

[xclerc]

I concur; the OS is probably not the cause, while the
bytecode-only nature of the switch is a prime suspect.

Out of curiosity, which version of dune are you using?
I am asking because core_bench expects dune >= 2.0
but your dune is likely not from the switch because of
ocaml/dune#3122 (which, I think, affects dune 2.x).

[fpottier]

I think I am using dune 2.2.0, from this switch.

$ which dune
/home/fpottier/.opam/4.09.0+bytecode-only/bin/dune
$ dune --version
2.2.0