re-compilation is crashed for project that have dep with option src_dirs set to [{"src", [{recursive, true}]}]

[AnufrievDE]

I have an umbrella project, with several apps/libs depends on each other. When I working with one of such lib, I'd like to compile/run it just inside its own folder, for debug purposes. I also use rebar3_path_deps plugin to be able to reference the dependencies via relative path like that:

{deps, [
  {task_server, {path, "../task_server"}}
]}.

{plugins, [
  rebar3_path_deps
]}.

So, I've noticed that when lib has the dependency with option {src_dirs, [{"src", [{recursive, true}]}]}, it can't be compiled the second time.

Environment

$ ❯ rebar3 report "rebar3 compile"
Rebar3 report
 version 3.16.1
 generated at 2021-06-16T22:44:36+00:00
=================
Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task: rebar3
Entered as:
  rebar3 compile
-----------------
Operating System: x86_64-unknown-linux-gnu
ERTS: Erlang/OTP 23 [erts-11.1.3] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe]
Root Directory: /home/ra1d3n17/.asdf/installs/erlang/23.1.3
Library directory: /home/ra1d3n17/.asdf/installs/erlang/23.1.3/lib
-----------------
Loaded Applications:
bbmustache: 1.10.0
certifi: 2.6.1
cf: 0.3.1
common_test: 1.19
compiler: 7.6.5
crypto: 4.8
cth_readable: 1.5.1
dialyzer: 4.2.1
edoc: 0.12
erlware_commons: 1.5.0
eunit: 2.6
eunit_formatters: 0.5.0
getopt: 1.0.1
hipe: 4.0.1
inets: 7.3
kernel: 7.1
providers: 1.8.1
public_key: 1.9.1
relx: 4.4.0
sasl: 4.0.1
snmp: 5.6.1
ssl_verify_fun: 1.1.6
stdlib: 3.13.2
syntax_tools: 2.3.1
tools: 3.4.1

-----------------
Escript path: undefined
Providers:
  app_discovery as clean compile compile cover ct deps dialyzer do edoc escriptize eunit get-deps help install install_deps list lock new path pkgs release relup report repos shell state tar tree unlock update upgrade upgrade upgrade version xref
...

Current behaviour

Re-compilation is crashed for project that have dep with option src_dirs set to [{"src", [{recursive, true}]}].

DIAGNOSTIC=1 rebar3 compile
===> Setting paths to [deps]
===> Compile (apps)
===> Setting paths to [plugins]
===> Setting paths to [deps]
===> Setting paths to [plugins]
===> Setting paths to [plugins]
===> Expanded command sequence to be run: [app_discovery,install_deps,lock,compile]
===> Running provider: app_discovery
===> Found top-level apps: [hackney_req]
        using config: [{src_dirs,["src"]},{lib_dirs,["apps/*","lib/*","."]}]
===> Uncaught error in rebar_core. Run with DIAGNOSTIC=1 to see stacktrace or consult rebar3.crashdump
===> Uncaught error: function_clause
===> Stack trace to the error location:
[{filename,join,
           ["/d/repos/qqq_umbrella/apps/hackney_req/_build/default/lib/task_server",
            {"src",[{recursive,true}]}],
           [{file,"filename.erl"},{line,446}]},
 {filename,join,1,[{file,"filename.erl"},{line,433}]},
 {rebar_app_discover,'-find_app_/5-lc$^1/1-1-',3,
                     [{file,"/home/runner/work/rebar3/rebar3/src/rebar_app_discover.erl"},
                      {line,351}]},
 {rebar_app_discover,'-find_app_/5-lc$^0/1-0-',3,
                     [{file,"/home/runner/work/rebar3/rebar3/src/rebar_app_discover.erl"},
                      {line,352}]},
 {rebar_app_discover,find_app_,5,
                     [{file,"/home/runner/work/rebar3/rebar3/src/rebar_app_discover.erl"},
                      {line,353}]},
 {rebar_app_info,discover,2,
                 [{file,"/home/runner/work/rebar3/rebar3/src/rebar_app_info.erl"},
                  {line,260}]},
 {rebar_app_utils,dep_to_app,7,
                  [{file,"/home/runner/work/rebar3/rebar3/src/rebar_app_utils.erl"},
                   {line,265}]},
 {rebar_app_utils,'-parse_deps/6-lc$^0/1-0-',6,
                  [{file,"/home/runner/work/rebar3/rebar3/src/rebar_app_utils.erl"},
                   {line,165}]}]
===> When submitting a bug report, please include the output of `rebar3 report "your command"`

Expected behaviour

Re-compilation should not crash.

[AnufrievDE]

Seems there is workaround with using {erl_opts, [{src_dirs,[{"src",[{recursive,true}]}]}]}. in dep lib, insted of {src_dirs,[{"src",[{recursive,true}]}]}.

[ferd]

See if #2582 fixes it.

[AnufrievDE]

#2582 fixes it (but with comment applied).

Tested both with "{src_dirs, [{"src", [{recursive, false}]}]}", "{src_dirs, [{"src", [{recursive, true}]}]}". Seems works as expected.

By the way, do you know is there any functional difference in compilation between specifing src_dirs option inside erl_opts and specifing src_dirs option directly in rebar.config?

[ferd]

Not really, aside from making things far more confusing on the implementation side. They should be considered the same in terms of effects, but it was part of a cleanup where initially it was supported in erl_opts and at some point we extracted it because that's far more of a general path config than a compiler options, since non-compiler tools should consider it useful as well.