[bug] Some build requires cannot be found in lockfile with conan 1.60.0 anymore

[emricolSICKAG]

Environment details

• Operating System+version: Windows
• Compiler+version: msvc (but no building required)
• Conan version: 1.60.0
• Python version: 3.8.6

Steps to reproduce

1. Create lock file containing build requires.
2. conan Install using the lock file

The installed component itself and 2 transitive components have the same build require.
The build require has another dependency (Doxygen).
When installing from lock files using conan 1.60.0 the conan install command fails with the following error message:
"ERROR: Require 'Doxygen' cannot be found in lockfile"
Installing with conan 1.59.0 works perfectly fine.

When I create the lock files with 1.59 and 1.60 the base lock files are identical.
When I create the lock file with a profile conan 1.60 changes the order of the failing build require.
However conan 1.60 fails to install both lock file versions.

Logs

"ERROR: Require 'Doxygen' cannot be found in lockfile"

[SzBosch]

I found a similar problem with lockfiles on conan 1.60.0 when build_requires have transitive dependencies.
Likely the root cause is the same.

Steps to reproduce:

1. create a ToolPkgA
2. create a ToolPkgB with a normal requirement to ToolPkgA
3. write a profile.txt with a build_require to ToolPkgB
4. write a empty conanfile.txt
5. conan lock create -pr:b=default -pr:h=$(pwd)/profile.txt conanfile.txt
6. conan info conanfile.txt -l conan.lock

Error:
The command on step 5. fails with complaining ToolPkgA not found in the lockfile (but it is already repsent).

Expectation:
Like in earlier conan versions, the info is printed containing both ToolPkgs without any error.

Can someone please confirm is issue?

Hint: I am already working on a fix, but first we need to have confirmation and a proper testcase for jenkins detecting this issue.

[stefansli]

I'm facing the same problem with tool requires and transitive dependencies (specifically the swig package as a tool_require).

[SzBosch]

@OEmrich, @vectorsli: Can you please try out my proposed fix 435bc08 and provide your feedback whether this works for you?

[stefansli]

@SzBosch it looks like this prevented the error. However I can't tell if the dependency graph is the same. At least generating new lockfiles produces the same lockfiles as with 1.59.0.

[memsharded]

For reference it seems #13788 was the PR introducing this change.
Let's see if it can be fixed without further issues, otherwise it might be better to revert it for some 1.60.X patch release.

If you all @OEmrich can test your projects against the #13902 changes, that would help a lot. We will try to add some further tests to it.

[SzBosch]

@memsharded: Without #13788 there is an other bug: It is not possible to have the same package twice in different contexts using lockfiles. Therfore you need to see the changes of #13788 and #13902 together.

To be honest #13788 alone should not have passed your automated regression tests as I would have expected this issue #13899 should have beed detected.
But it seems your automated regression tests lacks the scenario of transitive dependencies of a build_require together with lockfiles.

As we need both, having same package twice in different context AND transitive dependencies in build_require, using lockfiles in all cases, please to not revert the change. Please let me know when you need any support. Thank You.

[SzBosch]

Here in the documentation https://docs.conan.io/1/devtools/build_requires.html#build-and-host-contexts
there is this nice picture with my_pkg having dependencies to gtest & zlib in host context and having nasm and protobuf in build contect as well as the transitive dependency of protobuf to zlib.

E.g. this example should be a regression testcase, for with and without using lockfile.

[emricolSICKAG]

@SzBosch Your fix works perfectly fine. Thank you!

[czoido]

Closed by #13902 to be released in 1.60.1