diff --git a/changelog.rst b/changelog.rst index 8408da45f31a..2296bc6f3e88 100644 --- a/changelog.rst +++ b/changelog.rst @@ -6,6 +6,26 @@ For a more detailed description of the major changes that Conan 2.0 brings, comp 2.0.5 (18-May-2023) ------------------- +- Feature: `-v` argument defaults to the `VERBOSE` level. `#13839 `_ +- Feature: Avoid showing unnecessary skipped dependencies. Now, it only shows a list of reference names if exists skipped binaries. They can be completely listed by adding `-v` (verbose mode) to the current command. `#13836 `_ +- Feature: Allow step-into dependencies debugging for packages built locally with ``--build`` `#13833 `_ . Docs `here `__ +- Feature: Allow non relocatable, locally built packages with ``upload_policy="skip"`` and ``build_policy="missing"`` `#13833 `_ . Docs `here `__ +- Feature: Do not move "build" folders in cache when ``package-revision`` is computed to allow locating sources for dependencies debuggability with step-into `#13810 `_ +- Feature: New ``settings.possible_values()`` method to query the range of possible values for a setting. `#13796 `_ . Docs `here `__ +- Feature: Optimize and avoid hitting servers for binaries when ``upload_policy=skip`` `#13771 `_ +- Feature: Partially relativize generated environment .sh shell scripts `#13764 `_ +- Feature: Improve settings.yml error messages `#13748 `_ +- Feature: Auto create empty ``global.conf`` to improve UX looking for file in home. `#13746 `_ . Docs `here `__ +- Feature: Render the profile file name as profile_name `#13721 `_ . Docs `here `__ +- Feature: New global custom generators in cache "extensions/generators" that can be used by name. `#13718 `_ . Docs `here `__ +- Feature: Improve :command:`conan inspect` output, it now understands `set_name`/`set_version`. `#13716 `_ . Docs `here `__ +- Feature: Define new ``self.tool_requires("pkg/")`` to allow some tool-requires to follow and use the same version as the "host" regular requires do. `#13712 `_ . Docs `here `__ +- Feature: Introduce new ``core:skip_warns`` configuration to be able to silence some warnings in the output. `#13706 `_ . Docs `here `__ +- Feature: Add info_invalid to graph node serialization `#13688 `_ +- Feature: Computing and reporting the ``overrides`` in the graph, and in the ``graph build-order`` `#13680 `_ +- Feature: New ``revision_mode = "scm_folder"`` for mono-repo projects that want to use ``scm`` revisions. `#13562 `_ . Docs `here `__ +- Feature: Demonstrate that it is possible to ``tool_requires`` different versions of the same package. `#13529 `_ . Docs `here `__ +- Fix: `build_scripts` now set the `run` trait to `True` by default `#13901 `_ . Docs `here `__ - Fix: Fix XcodeDeps includes skipped dependencies. `#13880 `_ - Fix: Do not allow line feeds into ``pkg/version`` reference fields `#13870 `_ - Fix: Fix ``AutotoolsToolchain`` definition of ``tools.build:compiler_executable`` for Windows subsystems `#13867 `_ diff --git a/examples/extensions.rst b/examples/extensions.rst index c6187c8542f5..bd4478636938 100644 --- a/examples/extensions.rst +++ b/examples/extensions.rst @@ -7,4 +7,5 @@ Conan extensions examples :maxdepth: 2 extensions/commands/custom_commands + extensions/deployers/builtin_deployers extensions/deployers/custom_deployers diff --git a/examples/extensions/commands/clean/custom_command_clean_revisions.rst b/examples/extensions/commands/clean/custom_command_clean_revisions.rst index b10c6d2e3d75..e555d2bdc67e 100644 --- a/examples/extensions/commands/clean/custom_command_clean_revisions.rst +++ b/examples/extensions/commands/clean/custom_command_clean_revisions.rst @@ -3,6 +3,13 @@ Custom command: Clean old recipe and package revisions ====================================================== +.. note:: + + This is mostly an example command. The built-in ``conan remove *#!latest`` syntax, + meaning "all revisions but the latest" would probably be enough for this use case, + without needing this custom command. + + Please, first of all, clone the sources to recreate this project. You can find them in the `examples2.0 repository `_ in GitHub: @@ -173,6 +180,8 @@ Important classes to manage user input and user output: Conan public API ++++++++++++++++ +.. include:: ../../../../common/experimental_warning.inc + The most important part of this example is the usage of the Conan API via ``conan_api`` parameter. These are some examples which are being used in this custom command: diff --git a/examples/extensions/deployers/builtin_deployers.rst b/examples/extensions/deployers/builtin_deployers.rst new file mode 100644 index 000000000000..3d392f3fccc4 --- /dev/null +++ b/examples/extensions/deployers/builtin_deployers.rst @@ -0,0 +1,10 @@ +.. _examples_extensions_builtin_deployers: + + +Builtin deployers +================= + +.. toctree:: + :maxdepth: 2 + + dev/development_deploy diff --git a/examples/extensions/deployers/dev/development_deploy.rst b/examples/extensions/deployers/dev/development_deploy.rst new file mode 100644 index 000000000000..9456390737be --- /dev/null +++ b/examples/extensions/deployers/dev/development_deploy.rst @@ -0,0 +1,134 @@ +.. _examples_extensions_builtin_deployers_development: + + +Creating a Conan-agnostic deploy of dependencies for developer use +================================================================== + +With the ``full_deploy`` deployer it is possible to create a Conan-agnostic copy of dependencies that can be used by developers without even having Conan installed in their computers. + +Let's see it with an example. All the source code is in the +`examples2.0 Github repository `_ + +.. code-block:: bash + + $ git clone https://github.com/conan-io/examples2.git + $ cd examples2/examples/extensions/deployers/development_deploy + +In the folder we can find the following ``conanfile.txt``: + +.. code-block:: ini + + [requires] + zlib/1.2.13 + + [tool_requires] + cmake/3.25.3 + + [generators] + CMakeDeps + CMakeToolchain + + [layout] + cmake_layout + +The folder also contains a standard ``CMakeLists.txt`` and a ``main.cpp`` source file that can create +an executable that links with ``zlib`` library. + +We can install the Debug and Release dependencies, and deploy a local copy of the packages with: + +.. code-block:: bash + + $ conan install . --deploy=full_deploy --build=missing + $ conan install . --deploy=full_deploy -s build_type=Debug --build=missing + +This will create the following folders: + +.. code-block:: text + + ├──src + ├──build + │ ├──generators + | └── ZLibConfig.cmake + ├──full_deploy + │ ├──build + │ │ └──cmake + │ │ └──3.25.3 + │ │ └──x86_64 + │ │ ├──bin + │ │ + │ └──host + │ └──zlib + │ └──1.2.13 + │ ├──Debug + │ │ └──x86_64 + │ │ ├──include + │ │ ├──lib + │ └──Release + │ └──x86_64 + │ ├──include + │ ├──lib + + +This folder is fully self-contained. It contains both the necessary tools (like ``cmake`` executable), the headers and compiled libraries of ``zlib`` and the necessary files like ``ZLibConfig.cmake`` in the ``build/generators`` folder, that point to the binaries inside ``full_deploy`` with a relative path. + +The Conan cache can be removed, and even Conan uninstalled, then the folder could be moved elsewhere in the computer or copied to another computer, assuming it has the same configuration of OS, compiler, etc. + +.. code-block:: bash + + $ cd .. + $ cp -R development_deploy /some/other/place + $ cd /some/other/place + +And the files could be used by developers as: + +.. code-block:: bash + :caption: Windows + + $ cd build + # Activate the environment to use CMake 3.25 + $ generators\conanbuild.bat + $ cmake --version + cmake version 3.25.3 + # Configure, should match the settings used at install + $ cmake .. -G \"Visual Studio 17 2022\" -DCMAKE_TOOLCHAIN_FILE=generators/conan_toolchain.cmake + $ cmake --build . --config Release + $ Release\compressor.exe + ZLIB VERSION: 1.2.13 + + +The environment scripts in Linux and OSX are not relocatable, because they contain absolute paths and the ``sh`` shell `does not have any way to provide access to the current script directory for sourced files `_. + +This shouldn't be a big blocker, as a "search and replace" with ``sed`` in the generators folder can fix it: + +.. code-block:: bash + :caption: Linux + + $ cd build/Release/generators + # Fix folders in Linux + $ sed -i 's,{old_folder},{new_folder},g' * + # Fix folders in MacOS + $ sed -i '' 's,{old_folder},{new_folder},g' * + $ source conanbuild.sh + $ cd .. + $ cmake --version + cmake version 3.25.3 + $ cmake ../.. -DCMAKE_TOOLCHAIN_FILE=generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=Release + $ cmake --build . + $ ./compressor + ZLIB VERSION: 1.2.13 + + +.. note:: + + **Best practices** + + The fact that this flow is possible doesn't mean that it is recommended for the majority of cases. + It has some limitations: + + - It is less efficient, requiring an extra copy of dependencies + - Only ``CMakeDeps`` and ``CMakeToolchain`` are relocatable at this moment. For other build system integrations, please create a ticket in Github + - Linux and OSX shell scripts are not relocatable and require a manual ``sed`` + - The binary variability is limited to Release/Debug. The generated files are exclusively for the current configuration, changing any other setting (os, compiler, architecture) will require a different deploy + + In the general case, normal usage of the cache is recommended. This "relocatable development deployment" + could be useful for distributing final products that looks like an SDK, to consumers of a project not using Conan. diff --git a/examples/extensions/deployers/sources/custom_deployer_sources.rst b/examples/extensions/deployers/sources/custom_deployer_sources.rst index ab6656e4b14e..c58f5a1798da 100644 --- a/examples/extensions/deployers/sources/custom_deployer_sources.rst +++ b/examples/extensions/deployers/sources/custom_deployer_sources.rst @@ -31,7 +31,7 @@ Conan will look for the specified deployer in a few extra places in order, namel #. Absolute paths #. Relative to cwd -#. In the ``[CONAN_HOME]/extensions/deploy`` folder +#. In the ``[CONAN_HOME]/extensions/deployers`` folder #. Built-in deployers diff --git a/images/cheatsheet/conan2-cheatsheet-v5.pdf b/images/cheatsheet/conan2-cheatsheet-v5.pdf new file mode 100644 index 000000000000..2eee97c2ae80 Binary files /dev/null and b/images/cheatsheet/conan2-cheatsheet-v5.pdf differ diff --git a/images/cheatsheet/conan2-cheatsheet-v5.png b/images/cheatsheet/conan2-cheatsheet-v5.png new file mode 100644 index 000000000000..f5b879cf454e Binary files /dev/null and b/images/cheatsheet/conan2-cheatsheet-v5.png differ diff --git a/knowledge.rst b/knowledge.rst index 64c830741f7d..cc35428dc4da 100644 --- a/knowledge.rst +++ b/knowledge.rst @@ -8,6 +8,7 @@ Knowledge :maxdepth: 2 + knowledge/cheatsheet knowledge/guidelines knowledge/faq knowledge/videos diff --git a/knowledge/cheatsheet.rst b/knowledge/cheatsheet.rst new file mode 100644 index 000000000000..4da5c19ce0fe --- /dev/null +++ b/knowledge/cheatsheet.rst @@ -0,0 +1,16 @@ +*********** +Cheat sheet +*********** + +This is a visual cheat sheet for basic Conan commands and +concepts which users can print out and use as a handy reference. It is available +as both a PDF and PNG. + + :download:`PDF Format <../images/cheatsheet/conan2-cheatsheet-v5.pdf>` + + :download:`PNG Format <../images/cheatsheet/conan2-cheatsheet-v5.png>` + +.. image:: ../images/cheatsheet/conan2-cheatsheet-v5.png + :height: 600 px + :width: 800 px + :align: center diff --git a/reference/commands/install.rst b/reference/commands/install.rst index 812eaf4659e4..026ac7c0124f 100644 --- a/reference/commands/install.rst +++ b/reference/commands/install.rst @@ -256,7 +256,7 @@ There are 2 built-in deployers: Some generators might have the capability of redefining the target "package folder". That means that if some other generator like ``CMakeDeps`` is used that is pointing to the packages, it will be pointing to the local deployed -copy, and not to the original packages in the Conan cache. +copy, and not to the original packages in the Conan cache. See the full example in :ref:`examples_extensions_builtin_deployers_development`. It is also possible, and it is a powerful extension point, to write custom user deployers. Read more about custom deployers in :ref:`reference_extensions_deployers`. diff --git a/reference/extensions/custom_commands.rst b/reference/extensions/custom_commands.rst index e48369edfb77..8bde069435ec 100644 --- a/reference/extensions/custom_commands.rst +++ b/reference/extensions/custom_commands.rst @@ -1,7 +1,7 @@ .. _reference_commands_custom_commands: Custom commands -================= +=============== It's possible to create your own Conan commands to solve self-needs thanks to Python and Conan public API powers altogether. @@ -17,7 +17,7 @@ existing Conan one: :command:`conan your_command_name`. Scoping -++++++++++ ++++++++ It's possible to have another folder layer to group some commands under the same topic. @@ -87,7 +87,7 @@ Main decorator to declare a function as a new Conan command. Where the parameter conan_subcommand(formatters=None) -++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++ Similar to ``conan_command``, but this one is declaring a sub-command of an existing custom command. For instance: @@ -120,8 +120,53 @@ The command call looks like :command:`conan hello moon`. Notice that to declare a sub-command is required an empty Python function acts as the main command. -Formatters arguments -++++++++++++++++++++ +Argument definition and parsing +------------------------------- + +Commands can define their own arguments with the ``argparse`` Python library. + + +.. code-block:: python + + @conan_command(group='Creator') + def build(conan_api, parser, *args): + """ + Command help + """ + parser.add_argument("path", nargs="?", help='help for command') + add_reference_args(parser) + args = parser.parse_args(*args) + # Use args.path + + +When there are sub-commands, the base command cannot define arguments, only the sub-commands can do it. If you have a set of common arguments to all sub-commands, you can define a function that adds them. + +.. code-block:: python + + @conan_command(group="MyGroup") + def mycommand(conan_api, parser, *args): + """ + Command help + """ + # Do not define arguments in the base command + pass + + @conan_subcommand() + def mycommand_mysubcommand(conan_api: ConanAPI, parser, subparser, *args): + """ + Subcommand help + """ + # Arguments are added to "subparser" + subparser.add_argument("reference", help="Recipe reference or Package reference") + # You can add common args with your helper + # add_my_common_args(subparser) + # But parsing all of them happens to "parser" + args = parser.parse_args(*args) + # use args.reference + + +Formatters +---------- The return of the command will be passed as argument to the formatters. If there are different formatters that require different arguments, the approach is to return a dictionary, and let the formatters chose the @@ -152,8 +197,8 @@ arguments they need. For example, the ``graph info`` command uses several format "conan_api": conan_api} -Command function arguments ----------------------------- +Commands parameters +------------------- These are the passed arguments to any custom command and its sub-commands functions: diff --git a/reference/extensions/deployers.rst b/reference/extensions/deployers.rst index a9ec4c6d2b29..877312e0f3eb 100644 --- a/reference/extensions/deployers.rst +++ b/reference/extensions/deployers.rst @@ -37,6 +37,7 @@ Then every dependency will end up in a folder such as: ``[OUTPUT_FOLDER]/full_deploy/host/dep/0.1/Release/x86_64`` +See a full example of the usage of ``full_deploy`` deployer in :ref:`examples_extensions_builtin_deployers_development`. .. _reference_extensions_deployer_direct_deploy: @@ -62,7 +63,7 @@ Conan will look in these locations for the deployer in the following order: #. Absolute paths #. Relative to cwd -#. In the ``[CONAN_HOME]/extensions/deploy`` folder +#. In the ``[CONAN_HOME]/extensions/deployers`` folder #. As built-in deployers Conan will look for a ``deploy()`` method to call for each installed file. diff --git a/tutorial/consuming_packages/the_flexibility_of_conanfile_py.rst b/tutorial/consuming_packages/the_flexibility_of_conanfile_py.rst index 4032ea50028d..1bb7940d7f88 100644 --- a/tutorial/consuming_packages/the_flexibility_of_conanfile_py.rst +++ b/tutorial/consuming_packages/the_flexibility_of_conanfile_py.rst @@ -156,16 +156,20 @@ structure we want to follow and also to add some logic using Conan settings and Use the layout() method ----------------------- -In the previous examples, every time we executed a `conan install` command we had to use -the `--output-folder argument` to define where we wanted to create the files that Conan -generates. Also, note that we used a different folder when building in Windows or in -Linux/macOS depending if we were using a multi-config CMake generator or not. You can -define this directly in the `conanfile.py` inside the `layout()` method and make it work -for every platform without adding more changes: +In the previous examples, every time we executed a `conan install` command, we had to use +the `--output-folder` argument to define where we wanted to create the files that Conan +generates. There's a neater way to decide where we want Conan to generate the files for +the build system that will allow us to decide, for example, if we want different output +folders depending on the type of CMake generator we are using. You can define this +directly in the `conanfile.py` inside the `layout()` method and make it work for every +platform without adding more changes. + .. code-block:: python :caption: **conanfile.py** + import os + from conan import ConanFile @@ -182,23 +186,16 @@ for every platform without adding more changes: def layout(self): # We make the assumption that if the compiler is msvc the # CMake generator is multi-config - if self.settings.get_safe("compiler") == "msvc": - multi = True + multi = True if self.settings.get_safe("compiler") == "msvc" else False + if multi: + self.folders.generators = os.path.join("build", "generators") else: - multi = False - - self.folders.build = "build" if multi else f"build/{str(self.settings.build_type)}" - self.folders.generators = "build" + self.folders.generators = os.path.join("build", str(self.settings.build_type), "generators") -As you can see, we defined two different attributes for the Conanfile in the `layout()` method: - -* **self.folders.build** is the folder where the resulting binaries will be placed. The - location depends on the type of CMake generator. For multi-config, they will be located - in a dedicated folder inside the build folder, while for single-config, they will be - located directly in the build folder. -* **self.folders.generators** is the folder where all the auxiliary files generated by - Conan (CMake toolchain and cmake dependencies files) will be placed. +As you can see, we defined the **self.folders.generators** attribute in the `layout()` +method. This is the folder where all the auxiliary files generated by Conan (CMake +toolchain and cmake dependencies files) will be placed. Note that the definitions of the folders is different if it is a multi-config generator (like Visual Studio), or a single-config generator (like Unix Makefiles). In the @@ -216,9 +213,9 @@ Check that running the same commands as in the previous examples without the $ conan install . --build=missing $ cd build - $ conanbuild.bat + $ generators\conanbuild.bat # assuming Visual Studio 15 2017 is your VS version and that it matches your default profile - $ cmake .. -G "Visual Studio 15 2017" -DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake + $ cmake .. -G "Visual Studio 15 2017" -DCMAKE_TOOLCHAIN_FILE=generators\conan_toolchain.cmake $ cmake --build . --config Release ... Building with CMake version: 3.22.6 @@ -229,17 +226,17 @@ Check that running the same commands as in the previous examples without the Uncompressed size is: 233 Compressed size is: 147 ZLIB VERSION: 1.2.11 - $ deactivate_conanbuild.bat + $ generators\deactivate_conanbuild.bat .. code-block:: bash :caption: Linux, macOS $ conan install . --build=missing $ cd build - $ source conanbuild.sh + $ source ./Release/generators/conanbuild.sh Capturing current environment in deactivate_conanbuildenv-release-x86_64.sh Configuring environment variables - $ cmake .. -DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake -DCMAKE_BUILD_TYPE=Release + $ cmake .. -DCMAKE_TOOLCHAIN_FILE=Release/generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=Release $ cmake --build . ... Building with CMake version: 3.22.6 @@ -250,7 +247,7 @@ Check that running the same commands as in the previous examples without the Uncompressed size is: 233 Compressed size is: 147 ZLIB VERSION: 1.2.11 - $ source deactivate_conanbuild.sh + $ source ./Release/generators/deactivate_conanbuild.sh There's no need to always write this logic in the `conanfile.py`. There are some pre-defined layouts you can import and directly use in your recipe. For example, for the