[bug] ConanInvalidConfiguration doesn't raise exception when using conan install

[samuel-emrys]

Description

I'm running into an issue where it doesn't appear that ConanInvalidConfiguration exceptions aren't being raised when executing conan install, but are being raised when executing conan create. Is this inconsistent behaviour intentional or a bug? If it's intentional, what's the method of invalidating configurations when using conan install?

Environment Details (include every applicable attribute)

• Operating System+version: Ubuntu 18.04
• Compiler+version: GCC 7.5
• Conan version: 1.45.0
• Python version: 3.8.2

Steps to reproduce (Include if Applicable)

Add the following validate() definition to conanfile.py:

    def validate(self):
        print("validate() executed")
        raise ConanInvalidConfiguration("Invalid config")
        print("Exception not thrown")

Logs (Executed commands with output) (Include/Attach if Applicable)

conan install does not throw:

$ conan install . -if build -pr=gcc7

Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=gcc
compiler.libcxx=libstdc++11
compiler.version=7
os=Linux
os_build=Linux
[options]
[build_requires]
[env]
CC=/usr/bin/gcc-7
CXX=/usr/bin/g++-7
validate() executed
conanfile.py (example-conan-cmake-project/0.1.0): Installing package
Requirements
    armadillo/10.7.3 from 'conan-center' - Cache
    boost/1.78.0 from 'conan-center' - Cache
    bzip2/1.0.8 from 'conan-center' - Cache
    date/3.0.1 from 'conan-center' - Cache
    gsl-lite/0.39.0 from 'conan-center' - Cache
    hdf5/1.12.0 from 'conan-center' - Cache
    libbacktrace/cci.20210118 from 'conan-center' - Cache
    libcurl/7.78.0 from 'conan-center' - Cache
    openblas/0.3.15 from 'conan-center' - Cache
    openssl/1.1.1m from 'conan-center' - Cache
    zlib/1.2.11 from 'conan-center' - Cache
Packages
    armadillo/10.7.3:4bb7757412089122da7df8aa9e3a9ff00e6fa375 - Cache
    boost/1.78.0:cfef62e446e1bb07894de4523f2649afdfa94524 - Cache
    bzip2/1.0.8:b27e2b40bcb70be2e7d6f0e0c6e43e35c530f8d3 - Cache
    date/3.0.1:6f87c18eeaea5ff12c8d1afd2760afb6eed7b676 - Cache
    gsl-lite/0.39.0:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
    hdf5/1.12.0:e671ab4fbe82957d3db7f418a83efebd88ee9ca5 - Cache
    libbacktrace/cci.20210118:d50a0d523d98c15bb147b18fa7d203887c38be8b - Cache
    libcurl/7.78.0:45bcb8d0a291133fac1e6ef15bd9e100b5ea5a40 - Cache
    openblas/0.3.15:f93d34643c88c224fda3561071907c785870b96b - Cache
    openssl/1.1.1m:d50a0d523d98c15bb147b18fa7d203887c38be8b - Cache
    zlib/1.2.11:d50a0d523d98c15bb147b18fa7d203887c38be8b - Cache
Build requirements
    cmake/3.22.0 from 'conan-center' - Cache
    gtest/cci.20210126 from 'conan-center' - Cache
Build requirements packages
    cmake/3.22.0:5c09c752508b674ca5cb1f2d327b5a2d582866c8 - Cache
    gtest/cci.20210126:a4062ec0208a59375ac653551e662b6cc469fe58 - Cache

Installing (downloading, building) binaries...
bzip2/1.0.8: Already installed!
bzip2/1.0.8: Appending PATH environment variable: /home/user/.conan/data/bzip2/1.0.8/_/_/package/b27e2b40bcb70be2e7d6f0e0c6e43e35c530f8d3/bin
gsl-lite/0.39.0: Already installed!
gtest/cci.20210126: Already installed!
libbacktrace/cci.20210118: Already installed!
openblas/0.3.15: Already installed!
openblas/0.3.15: Setting OpenBLAS_HOME environment variable: /home/user/.conan/data/openblas/0.3.15/_/_/package/f93d34643c88c224fda3561071907c785870b96b
openssl/1.1.1m: Already installed!
zlib/1.2.11: Already installed!
boost/1.78.0: Already installed!
cmake/3.22.0: Already installed!
cmake/3.22.0: Appending PATH environment variable: /home/user/.conan/data/cmake/3.22.0/_/_/package/5c09c752508b674ca5cb1f2d327b5a2d582866c8/bin
hdf5/1.12.0: Already installed!
libcurl/7.78.0: Already installed!
armadillo/10.7.3: Already installed!
date/3.0.1: Already installed!
conanfile.py (example-conan-cmake-project/0.1.0): Applying build-requirement: cmake/3.22.0
conanfile.py (example-conan-cmake-project/0.1.0): Applying build-requirement: gtest/cci.20210126
conanfile.py (example-conan-cmake-project/0.1.0): Generator cmake_find_package created Findcmake.cmake
conanfile.py (example-conan-cmake-project/0.1.0): Generator cmake_find_package created FindGTest.cmake
conanfile.py (example-conan-cmake-project/0.1.0): Generator cmake_find_package created Finddate.cmake
conanfile.py (example-conan-cmake-project/0.1.0): Generator cmake_find_package created Findarmadillo.cmake
conanfile.py (example-conan-cmake-project/0.1.0): Generator cmake_find_package created FindBoost.cmake
conanfile.py (example-conan-cmake-project/0.1.0): Generator cmake_find_package created Findgsl-lite.cmake
conanfile.py (example-conan-cmake-project/0.1.0): Generator cmake_find_package created FindCURL.cmake
conanfile.py (example-conan-cmake-project/0.1.0): Generator cmake_find_package created FindHDF5.cmake
conanfile.py (example-conan-cmake-project/0.1.0): Generator cmake_find_package created FindOpenBLAS.cmake
conanfile.py (example-conan-cmake-project/0.1.0): Generator cmake_find_package created FindBZip2.cmake
conanfile.py (example-conan-cmake-project/0.1.0): Generator cmake_find_package created Findlibbacktrace.cmake
conanfile.py (example-conan-cmake-project/0.1.0): Generator cmake_find_package created FindOpenSSL.cmake
conanfile.py (example-conan-cmake-project/0.1.0): Generator cmake_find_package created FindZLIB.cmake
conanfile.py (example-conan-cmake-project/0.1.0): Generator cmake created conanbuildinfo.cmake
conanfile.py (example-conan-cmake-project/0.1.0): Generator txt created conanbuildinfo.txt
conanfile.py (example-conan-cmake-project/0.1.0): Aggregating env generators
conanfile.py (example-conan-cmake-project/0.1.0): Generated conaninfo.txt
conanfile.py (example-conan-cmake-project/0.1.0): Generated graphinfo

conan create does throw:

$ conan create . -pr=gcc7

Exporting package recipe
example-conan-cmake-project/0.1.0 exports_sources: Copied 47 '.txt' files
example-conan-cmake-project/0.1.0 exports_sources: Copied 4 '.cpp' files: main.cpp, foo.cpp, bar.cpp, test_foo.cpp
example-conan-cmake-project/0.1.0 exports_sources: Copied 3 '.h' files: foo.h, bar.h, baz.h
example-conan-cmake-project/0.1.0 exports_sources: Copied 6 '.cmake' files
example-conan-cmake-project/0.1.0 exports_sources: Copied 132 files
example-conan-cmake-project/0.1.0 exports_sources: Copied 2 '.cfg' files: pyvenv.cfg, sysconfig.cfg
example-conan-cmake-project/0.1.0 exports_sources: Copied 1388 '.py' files
example-conan-cmake-project/0.1.0 exports_sources: Copied 2 '.pth' files: _virtualenv.pth, distutils-precedence.pth
example-conan-cmake-project/0.1.0 exports_sources: Copied 3 '.virtualenv' files: pip-21.3.1.virtualenv, wheel-0.37.0.virtualenv, setuptools-58.3.0.virtualenv
example-conan-cmake-project/0.1.0 exports_sources: Copied 1217 '.pyc' files
example-conan-cmake-project/0.1.0 exports_sources: Copied 2 '.tmpl' files: script (dev).tmpl, script.tmpl
example-conan-cmake-project/0.1.0 exports_sources: Copied 14 '.exe' files
example-conan-cmake-project/0.1.0 exports_sources: Copied 1 '.xml' file: launcher manifest.xml
example-conan-cmake-project/0.1.0 exports_sources: Copied 2 '.rst' files: LICENSE.rst, LICENSE.rst
example-conan-cmake-project/0.1.0 exports_sources: Copied 5 '.typed' files
example-conan-cmake-project/0.1.0 exports_sources: Copied 2 '.pem' files: cacert.pem, cacert.pem
example-conan-cmake-project/0.1.0 exports_sources: Copied 2 '.so' files: _yaml.cpython-38-x86_64-linux-gnu.so, _speedups.cpython-38-x86_64-linux-gnu.so
example-conan-cmake-project/0.1.0 exports_sources: Copied 1 '.pyi' file: _speedups.pyi
example-conan-cmake-project/0.1.0 exports_sources: Copied 1 '.c' file: _speedups.c
example-conan-cmake-project/0.1.0 exports_sources: Copied 2 '.md' files: LICENSE.md, LICENSE.md
example-conan-cmake-project/0.1.0 exports_sources: Copied 1 '.sh' file: completion.sh
example-conan-cmake-project/0.1.0 exports_sources: Copied 1 '.1' file: tqdm.1
example-conan-cmake-project/0.1.0 exports_sources: Copied 1 '.gz' file: dateutil-zoneinfo.tar.gz
example-conan-cmake-project/0.1.0 exports_sources: Copied 5 '.8' files
example-conan-cmake-project/0.1.0 exports_sources: Copied 1 '.fish' file: activate.fish
example-conan-cmake-project/0.1.0 exports_sources: Copied 1 '.ps1' file: activate.ps1
example-conan-cmake-project/0.1.0 exports_sources: Copied 2 '.nu' files: deactivate.nu, activate.nu
example-conan-cmake-project/0.1.0 exports_sources: Copied 1 '.csh' file: activate.csh
example-conan-cmake-project/0.1.0: A new conanfile.py version was exported
example-conan-cmake-project/0.1.0: Folder: /home/user/.conan/data/example-conan-cmake-project/0.1.0/_/_/export
example-conan-cmake-project/0.1.0: Using the exported files summary hash as the recipe revision: 5d3d2646ca6be04b03cceeeca56ff0c6
example-conan-cmake-project/0.1.0: Package recipe modified in export, forcing source folder removal
example-conan-cmake-project/0.1.0: Use the --keep-source, -k option to skip it
example-conan-cmake-project/0.1.0: Removing the local binary packages from different recipe revisions
example-conan-cmake-project/0.1.0: Exported revision: 5d3d2646ca6be04b03cceeeca56ff0c6
Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=gcc
compiler.libcxx=libstdc++11
compiler.version=7
os=Linux
os_build=Linux
[options]
[build_requires]
[env]
CC=/usr/bin/gcc-7
CXX=/usr/bin/g++-7
validate() executed
example-conan-cmake-project/0.1.0 (test package): Installing package
Requirements
    armadillo/10.7.3 from 'conan-center' - Cache
    boost/1.78.0 from 'conan-center' - Cache
    bzip2/1.0.8 from 'conan-center' - Cache
    date/3.0.1 from 'conan-center' - Cache
    example-conan-cmake-project/0.1.0 from local cache - Cache
    gsl-lite/0.39.0 from 'conan-center' - Cache
    hdf5/1.12.0 from 'conan-center' - Cache
    libbacktrace/cci.20210118 from 'conan-center' - Cache
    libcurl/7.78.0 from 'conan-center' - Cache
    openblas/0.3.15 from 'conan-center' - Cache
    openssl/1.1.1m from 'conan-center' - Cache
    zlib/1.2.11 from 'conan-center' - Cache
Packages
    armadillo/10.7.3:4bb7757412089122da7df8aa9e3a9ff00e6fa375 - Cache
    boost/1.78.0:cfef62e446e1bb07894de4523f2649afdfa94524 - Cache
    bzip2/1.0.8:b27e2b40bcb70be2e7d6f0e0c6e43e35c530f8d3 - Cache
    date/3.0.1:6f87c18eeaea5ff12c8d1afd2760afb6eed7b676 - Cache
    example-conan-cmake-project/0.1.0:INVALID - Invalid
    gsl-lite/0.39.0:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
    hdf5/1.12.0:e671ab4fbe82957d3db7f418a83efebd88ee9ca5 - Cache
    libbacktrace/cci.20210118:d50a0d523d98c15bb147b18fa7d203887c38be8b - Cache
    libcurl/7.78.0:45bcb8d0a291133fac1e6ef15bd9e100b5ea5a40 - Cache
    openblas/0.3.15:f93d34643c88c224fda3561071907c785870b96b - Cache
    openssl/1.1.1m:d50a0d523d98c15bb147b18fa7d203887c38be8b - Cache
    zlib/1.2.11:d50a0d523d98c15bb147b18fa7d203887c38be8b - Cache

Installing (downloading, building) binaries...
ERROR: There are invalid packages (packages that cannot exist for this configuration):
example-conan-cmake-project/0.1.0: Invalid ID: Invalid config

[memsharded]

Hi @samuel-emrys

This is kind of expected, because the validate() method only runs for packages, not for consumers.
We could have a look and see if it would be possible to run it for consumers, lets try.

[memsharded]

#10725 tries this, but sounds a bit risky for 1.X, we will analyze it.

[memsharded]

#10725 was merged, but it is displaying only a warning, not a hard error, because there are situations that could make sense (and it could be breaking), for example, installing some dependencies for a system to see if able to build locally, before removing the validate() exceptions.

[memsharded]

Merged to 2.0 raising exception.

[samuel-emrys]

#10725 was merged, but it is displaying only a warning, not a hard error, because there are situations that could make sense (and it could be breaking), for example, installing some dependencies for a system to see if able to build locally, before removing the validate() exceptions.

Hi @memsharded, are you able to elaborate on this use case? From what you've described, that's exactly the use case we would be trying to prevent - ability to say "No, this package can't be installed on this operating system" (or any other criteria) and a manual attempt to do so would fail. Why does this need to be a warning and not a hard error?

[memsharded]

Hi @memsharded, are you able to elaborate on this use case? From what you've described, that's exactly the use case we would be trying to prevent - ability to say "No, this package can't be installed on this operating system" (or any other criteria) and a manual attempt to do so would fail. Why does this need to be a warning and not a hard error?

Yes, lets say that you have a package validate() that is raising for cppstd<17 configuration.
There might still be some users that are doing a conan install . -s compiler.cppstd=11, because that might be still work, and succesfully download and install the dependencies in built with cppstd=11. Maybe it won't be able to build itself with cppstd 11, but it will manually inject or change the cppstd directly in the consumer build system scripts.

So just in case some users are already relying on this behavior, we are not raising in 1.X but just warning, and then raising in 2.0 (and the solution there will be to modify the validate() first, but we cannot request this in 1.X because it will be breaking for some flows).

[HappySeaFox]

Seems to be related to conan-io/conan-center-index#13001