Merge pull request #3237 from conan-io/release/2.0

Update develop2 with Release/2.0

Author: czoido

changelog.rst

@@ -6,6 +6,26 @@ For a more detailed description of the major changes that Conan 2.0 brings, comp
 2.0.5 (18-May-2023)
 -------------------
 
+- Feature: `-v` argument defaults to the `VERBOSE` level. `#13839 <https://github.com/conan-io/conan/pull/13839>`_
+- Feature: Avoid showing unnecessary skipped dependencies. Now, it only shows a list of reference names if exists skipped binaries. They can be completely listed by adding `-v` (verbose mode) to the current command. `#13836 <https://github.com/conan-io/conan/pull/13836>`_
+- Feature: Allow step-into dependencies debugging for packages built locally with ``--build`` `#13833 <https://github.com/conan-io/conan/pull/13833>`_ . Docs `here <https://github.com/conan-io/docs/pull/3210>`__
+- Feature: Allow non relocatable, locally built packages with ``upload_policy="skip"`` and ``build_policy="missing"`` `#13833 <https://github.com/conan-io/conan/pull/13833>`_ . Docs `here <https://github.com/conan-io/docs/pull/3210>`__
+- Feature: Do not move "build" folders in cache when ``package-revision`` is computed to allow locating sources for dependencies debuggability with step-into `#13810 <https://github.com/conan-io/conan/pull/13810>`_
+- Feature: New ``settings.possible_values()`` method to query the range of possible values for a setting. `#13796 <https://github.com/conan-io/conan/pull/13796>`_ . Docs `here <https://github.com/conan-io/docs/pull/3212>`__
+- Feature: Optimize and avoid hitting servers for binaries when ``upload_policy=skip`` `#13771 <https://github.com/conan-io/conan/pull/13771>`_
+- Feature: Partially relativize generated environment .sh shell scripts `#13764 <https://github.com/conan-io/conan/pull/13764>`_
+- Feature: Improve settings.yml error messages `#13748 <https://github.com/conan-io/conan/pull/13748>`_
+- Feature: Auto create empty ``global.conf`` to improve UX looking for file in home. `#13746 <https://github.com/conan-io/conan/pull/13746>`_ . Docs `here <https://github.com/conan-io/docs/pull/3211>`__
+- Feature: Render the profile file name as profile_name `#13721 <https://github.com/conan-io/conan/pull/13721>`_ . Docs `here <https://github.com/conan-io/docs/pull/3180>`__
+- Feature: New global custom generators in cache "extensions/generators" that can be used by name. `#13718 <https://github.com/conan-io/conan/pull/13718>`_ . Docs `here <https://github.com/conan-io/docs/pull/3213>`__
+- Feature: Improve :command:`conan inspect` output, it now understands `set_name`/`set_version`. `#13716 <https://github.com/conan-io/conan/pull/13716>`_ . Docs `here <https://github.com/conan-io/docs/pull/3204>`__
+- Feature: Define new ``self.tool_requires("pkg/<host_version>")`` to allow some tool-requires to follow and use the same version as the "host" regular requires do. `#13712 <https://github.com/conan-io/conan/pull/13712>`_ . Docs `here <https://github.com/conan-io/docs/pull/3223>`__
+- Feature: Introduce new ``core:skip_warns`` configuration to be able to silence some warnings in the output. `#13706 <https://github.com/conan-io/conan/pull/13706>`_ . Docs `here <https://github.com/conan-io/docs/pull/3215>`__
+- Feature: Add info_invalid to graph node serialization `#13688 <https://github.com/conan-io/conan/pull/13688>`_
+- Feature: Computing and reporting the ``overrides`` in the graph, and in the ``graph build-order`` `#13680 <https://github.com/conan-io/conan/pull/13680>`_
+- Feature: New ``revision_mode = "scm_folder"`` for mono-repo projects that want to use ``scm`` revisions. `#13562 <https://github.com/conan-io/conan/pull/13562>`_ . Docs `here <https://github.com/conan-io/docs/pull/3218>`__
+- Feature: Demonstrate that it is possible to ``tool_requires`` different versions of the same package. `#13529 <https://github.com/conan-io/conan/pull/13529>`_ . Docs `here <https://github.com/conan-io/docs/pull/3219>`__
+- Fix: `build_scripts` now set the `run` trait to `True` by default `#13901 <https://github.com/conan-io/conan/pull/13901>`_ . Docs `here <https://github.com/conan-io/docs/pull/3206>`__
 - Fix: Fix XcodeDeps includes skipped dependencies. `#13880 <https://github.com/conan-io/conan/pull/13880>`_
 - Fix: Do not allow line feeds into ``pkg/version`` reference fields `#13870 <https://github.com/conan-io/conan/pull/13870>`_
 - Fix: Fix ``AutotoolsToolchain`` definition of  ``tools.build:compiler_executable`` for Windows subsystems `#13867 <https://github.com/conan-io/conan/pull/13867>`_

examples/extensions.rst

@@ -7,4 +7,5 @@ Conan extensions examples
    :maxdepth: 2
 
    extensions/commands/custom_commands
+   extensions/deployers/builtin_deployers
    extensions/deployers/custom_deployers

examples/extensions/commands/clean/custom_command_clean_revisions.rst

@@ -3,6 +3,13 @@
 Custom command: Clean old recipe and package revisions
 ======================================================
 
+.. note::
+
+    This is mostly an example command. The built-in ``conan remove *#!latest`` syntax,
+    meaning "all revisions but the latest" would probably be enough for this use case,
+    without needing this custom command.
+
+
 Please, first of all, clone the sources to recreate this project. You can find them in the
 `examples2.0 repository <https://github.com/conan-io/examples2>`_ in GitHub:
 
@@ -173,6 +180,8 @@ Important classes to manage user input and user output:
 Conan public API
 ++++++++++++++++
 
+.. include:: ../../../../common/experimental_warning.inc
+
 The most important part of this example is the usage of the Conan API via ``conan_api`` parameter. These are some examples
 which are being used in this custom command:
 

examples/extensions/deployers/builtin_deployers.rst

@@ -0,0 +1,10 @@
+.. _examples_extensions_builtin_deployers:
+
+
+Builtin deployers
+=================
+
+.. toctree::
+   :maxdepth: 2
+
+   dev/development_deploy

examples/extensions/deployers/dev/development_deploy.rst

@@ -0,0 +1,134 @@
+.. _examples_extensions_builtin_deployers_development:
+
+
+Creating a Conan-agnostic deploy of dependencies for developer use
+==================================================================
+
+With the ``full_deploy`` deployer it is possible to create a Conan-agnostic copy of dependencies that can be used by developers without even having Conan installed in their computers.
+
+Let's see it with an example. All the source code is in the
+`examples2.0 Github repository <https://github.com/conan-io/examples2>`_
+
+.. code-block:: bash
+
+    $ git clone https://github.com/conan-io/examples2.git
+    $ cd examples2/examples/extensions/deployers/development_deploy
+
+In the folder we can find the following ``conanfile.txt``:
+
+.. code-block:: ini
+
+    [requires]
+    zlib/1.2.13
+
+    [tool_requires]
+    cmake/3.25.3
+
+    [generators]
+    CMakeDeps
+    CMakeToolchain
+
+    [layout]
+    cmake_layout
+
+The folder also contains a standard ``CMakeLists.txt`` and a ``main.cpp`` source file that can create
+an executable that links with ``zlib`` library.
+
+We can install the Debug and Release dependencies, and deploy a local copy of the packages with:
+
+.. code-block:: bash
+
+    $ conan install . --deploy=full_deploy --build=missing
+    $ conan install . --deploy=full_deploy -s build_type=Debug --build=missing
+
+This will create the following folders:
+
+.. code-block:: text
+
+    ├──src
+    ├──build
+    │   ├──generators
+    |         └── ZLibConfig.cmake
+    ├──full_deploy
+    │   ├──build
+    │   │   └──cmake
+    │   │       └──3.25.3
+    │   │           └──x86_64
+    │   │               ├──bin
+    │   │
+    │   └──host
+    │       └──zlib
+    │           └──1.2.13
+    │               ├──Debug
+    │               │   └──x86_64
+    │               │       ├──include
+    │               │       ├──lib
+    │               └──Release
+    │                   └──x86_64
+    │                       ├──include
+    │                       ├──lib
+
+
+This folder is fully self-contained. It contains both the necessary tools (like ``cmake`` executable), the headers and compiled libraries of ``zlib`` and the necessary files like ``ZLibConfig.cmake`` in the ``build/generators`` folder, that point to the binaries inside ``full_deploy`` with a relative path. 
+
+The Conan cache can be removed, and even Conan uninstalled, then the folder could be moved elsewhere in the computer or copied to another computer, assuming it has the same configuration of OS, compiler, etc.
+
+.. code-block:: bash
+
+    $ cd ..
+    $ cp -R development_deploy /some/other/place
+    $ cd /some/other/place
+
+And the files could be used by developers as:
+
+.. code-block:: bash
+    :caption: Windows
+
+    $ cd build
+    # Activate the environment to use CMake 3.25
+    $ generators\conanbuild.bat
+    $ cmake --version
+    cmake version 3.25.3
+    # Configure, should match the settings used at install
+    $ cmake .. -G \"Visual Studio 17 2022\" -DCMAKE_TOOLCHAIN_FILE=generators/conan_toolchain.cmake
+    $ cmake --build . --config Release
+    $ Release\compressor.exe
+    ZLIB VERSION: 1.2.13
+
+
+The environment scripts in Linux and OSX are not relocatable, because they contain absolute paths and the ``sh`` shell `does not have any way to provide access to the current script directory for sourced files <https://stackoverflow.com/questions/29832037/how-to-get-script-directory-in-posix-sh/29835459#29835459>`_.
+
+This shouldn't be a big blocker, as a "search and replace" with ``sed`` in the generators folder can fix it:
+
+.. code-block:: bash
+    :caption: Linux
+
+    $ cd build/Release/generators
+    # Fix folders in Linux
+    $ sed -i 's,{old_folder},{new_folder},g' *
+    # Fix folders in MacOS
+    $ sed -i '' 's,{old_folder},{new_folder},g' *
+    $ source conanbuild.sh
+    $ cd ..
+    $ cmake --version
+    cmake version 3.25.3
+    $ cmake ../.. -DCMAKE_TOOLCHAIN_FILE=generators/conan_toolchain.cmake -DCMAKE_BUILD_TYPE=Release
+    $ cmake --build .
+    $ ./compressor
+    ZLIB VERSION: 1.2.13
+
+
+.. note::
+
+    **Best practices**
+
+    The fact that this flow is possible doesn't mean that it is recommended for the majority of cases.
+    It has some limitations:
+
+    - It is less efficient, requiring an extra copy of dependencies
+    - Only ``CMakeDeps`` and ``CMakeToolchain`` are relocatable at this moment. For other build system integrations, please create a ticket in Github
+    - Linux and OSX shell scripts are not relocatable and require a manual ``sed``
+    - The binary variability is limited to Release/Debug. The generated files are exclusively for the current configuration, changing any other setting (os, compiler, architecture) will require a different deploy
+
+    In the general case, normal usage of the cache is recommended. This "relocatable development deployment"
+    could be useful for distributing final products that looks like an SDK, to consumers of a project not using Conan.

examples/extensions/deployers/sources/custom_deployer_sources.rst

@@ -31,7 +31,7 @@ Conan will look for the specified deployer in a few extra places in order, namel
 
 #. Absolute paths
 #. Relative to cwd
-#. In the ``[CONAN_HOME]/extensions/deploy`` folder
+#. In the ``[CONAN_HOME]/extensions/deployers`` folder
 #. Built-in deployers
 
 

images/cheatsheet/conan2-cheatsheet-v5.pdf

@@ -8,6 +8,7 @@ Knowledge
    :maxdepth: 2
 
 
+   knowledge/cheatsheet
    knowledge/guidelines
    knowledge/faq
    knowledge/videos

images/cheatsheet/conan2-cheatsheet-v5.png

@@ -0,0 +1,16 @@
+***********
+Cheat sheet
+***********
+
+This is a visual cheat sheet for basic Conan commands and
+concepts which users can print out and use as a handy reference. It is available
+as both a PDF and PNG.
+
+  :download:`PDF Format <../images/cheatsheet/conan2-cheatsheet-v5.pdf>`
+
+  :download:`PNG Format <../images/cheatsheet/conan2-cheatsheet-v5.png>`
+  
+.. image:: ../images/cheatsheet/conan2-cheatsheet-v5.png
+   :height: 600 px 
+   :width: 800 px 
+   :align: center

knowledge.rst

@@ -256,7 +256,7 @@ There are 2 built-in deployers:
 
 Some generators might have the capability of redefining the target "package folder". That means that if some other
 generator like ``CMakeDeps`` is used that is pointing to the packages, it will be pointing to the local deployed
-copy, and not to the original packages in the Conan cache.
+copy, and not to the original packages in the Conan cache. See the full example in :ref:`examples_extensions_builtin_deployers_development`.
 
 It is also possible, and it is a powerful extension point, to write custom user deployers.
 Read more about custom deployers in :ref:`reference_extensions_deployers`.

knowledge/cheatsheet.rst

@@ -1,7 +1,7 @@
 .. _reference_commands_custom_commands:
 
 Custom commands
-=================
+===============
 
 It's possible to create your own Conan commands to solve self-needs thanks to Python and Conan public API powers altogether.
 
@@ -17,7 +17,7 @@ existing Conan one: :command:`conan your_command_name`.
 
 
 Scoping
-++++++++++
++++++++
 
 It's possible to have another folder layer to group some commands under the same topic.
 
@@ -87,7 +87,7 @@ Main decorator to declare a function as a new Conan command. Where the parameter
 
 
 conan_subcommand(formatters=None)
-++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++
 
 Similar to ``conan_command``, but this one is declaring a sub-command of an existing custom command. For instance:
 
@@ -120,8 +120,53 @@ The command call looks like :command:`conan hello moon`.
     Notice that to declare a sub-command is required an empty Python function acts as the main command.
 
 
-Formatters arguments
-++++++++++++++++++++
+Argument definition and parsing
+-------------------------------
+
+Commands can define their own arguments with the ``argparse`` Python library.
+
+
+.. code-block:: python
+    
+    @conan_command(group='Creator')
+    def build(conan_api, parser, *args):
+        """
+        Command help
+        """
+        parser.add_argument("path", nargs="?", help='help for command')
+        add_reference_args(parser)
+        args = parser.parse_args(*args)
+        # Use args.path
+
+
+When there are sub-commands, the base command cannot define arguments, only the sub-commands can do it. If you have a set of common arguments to all sub-commands, you can define a function that adds them.
+
+.. code-block:: python
+
+    @conan_command(group="MyGroup")
+    def mycommand(conan_api, parser, *args):
+        """
+        Command help
+        """
+        # Do not define arguments in the base command
+        pass
+
+    @conan_subcommand()
+    def mycommand_mysubcommand(conan_api: ConanAPI, parser, subparser, *args):
+        """
+        Subcommand help
+        """
+        # Arguments are added to "subparser"
+        subparser.add_argument("reference", help="Recipe reference or Package reference")
+        # You can add common args with your helper
+        # add_my_common_args(subparser)
+        # But parsing all of them happens to "parser"
+        args = parser.parse_args(*args)
+        # use args.reference
+
+
+Formatters
+----------
 
 The return of the command will be passed as argument to the formatters. If there are different formatters that 
 require different arguments, the approach is to return a dictionary, and let the formatters chose the 
@@ -152,8 +197,8 @@ arguments they need. For example, the ``graph info`` command uses several format
                 "conan_api": conan_api}
 
 
-Command function arguments
-----------------------------
+Commands parameters
+-------------------
 
 These are the passed arguments to any custom command and its sub-commands functions:
 

reference/commands/install.rst

@@ -37,6 +37,7 @@ Then every dependency will end up in a folder such as:
 
 ``[OUTPUT_FOLDER]/full_deploy/host/dep/0.1/Release/x86_64``
 
+See a full example of the usage of ``full_deploy`` deployer in :ref:`examples_extensions_builtin_deployers_development`.
 
 .. _reference_extensions_deployer_direct_deploy:
 
@@ -62,7 +63,7 @@ Conan will look in these locations for the deployer in the following order:
 
 #. Absolute paths
 #. Relative to cwd
-#. In the ``[CONAN_HOME]/extensions/deploy`` folder
+#. In the ``[CONAN_HOME]/extensions/deployers`` folder
 #. As built-in deployers
 
 Conan will look for a ``deploy()`` method to call for each installed file.