Skip to content

Docs about 'scm_to_conandata' #1522

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 15, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 40 additions & 26 deletions creating_packages/package_repo.rst
Original file line number Diff line number Diff line change
Expand Up @@ -108,44 +108,58 @@ When you export the recipe (or when :command:`conan create` is called) the expor
remote and commit of the local repository:

.. code-block:: python
:emphasize-lines: 7, 8
:emphasize-lines: 8-10

import os
from conans import ConanFile, CMake, tools

class HelloConan(ConanFile):
scm = {
"type": "git", # Use "type": "svn", if local repo is managed using SVN
"subfolder": "hello",
"url": "auto",
"revision": "auto"
"revision": "auto",
"password": os.environ.get("SECRET", None)
}
...

You can commit and push the *conanfile.py* to your origin repository, which will always preserve the ``auto``
values. But when the file is exported to the Conan local cache (except you have uncommitted changes, read below),
the copied recipe in the local cache will point to the captured remote and commit:

.. code-block:: python
:emphasize-lines: 7, 8

from conans import ConanFile, CMake, tools

class HelloConan(ConanFile):
scm = {
"type": "git",
"subfolder": "hello",
"url": "https://github.com/conan-io/hello.git",
"revision": "437676e15da7090a1368255097f51b1a470905a0"
}
...

So when you :ref:`upload the recipe <uploading_packages>` to a Conan remote, the recipe will contain
the "resolved" URL and commit.

When you are requiring your ``HelloConan``, the :command:`conan install` will retrieve the recipe from the
remote. If you are building the package, the source code will be fetched from the captured url/commit.

As SCM attributes are evaluated in the workspace context (see :ref:`scm attribute <scm_attribute>`),
values. When the file is exported to the Conan local cache (except you have uncommitted changes, read below),
these data will be stored in the *conanfile.py* itself (Conan will modify the file) or in a special file
:ref:`conandata_yml` that will be stored together with the recipe, depending on the value of the configuration
parameter :ref:`scm_to_conandata<conan_conf>`.

* If the ``scm_to_conandata`` is not activated (default behavior in Conan v1.x) Conan will store a modified
version of the *conanfile.py* with the values of the fields in plain text:

.. code-block:: python
:emphasize-lines: 8-10

import os
from conans import ConanFile, CMake, tools

class HelloConan(ConanFile):
scm = {
"type": "git",
"subfolder": "hello",
"url": "https://github.com/conan-io/hello.git",
"revision": "437676e15da7090a1368255097f51b1a470905a0",
"password": "MY_SECRET"
}
...

So when you :ref:`upload the recipe <uploading_packages>` to a Conan remote, the recipe will contain
the "resolved" URL and commit.

* If ``scm_to_conandata`` is activated, the value of these fields (except ``username`` and ``password``) will
be stored in the :ref:`conandata_yml` file that will be automatically exported with the recipe.

Whichever option you choose, the data resolved will be asigned by Conan to the corresponding field when the recipe
file is loaded, and they will be available for all the methods defined in the recipe. Also, if building the package
from sources, Conan will fetch the code in the captured url/commit before running the method ``source()`` in the
recipe (if defined).

As SCM attributes are evaluated in the local directory context (see :ref:`scm attribute <scm_attribute>`),
you can write more complex functions to retrieve the proper values, this source *conanfile.py* will
be valid too:

Expand Down
17 changes: 13 additions & 4 deletions reference/conanfile/attributes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1187,12 +1187,21 @@ Used to clone/checkout a repository. It is a dictionary with the following possi
- ``shallow``: Will sync the git submodules using ``submodule sync``
- ``recursive``: Will sync the git submodules using ``submodule sync --recursive``

SCM attributes are evaluated in the workspace context where the *conanfile.py* is located before
SCM attributes are evaluated in the working directory where the *conanfile.py* is located before
exporting it to the Conan cache, so these values can be returned from arbitrary functions that
depend on the workspace layout. Nevertheless, all the other code in the recipe must be able to
depend on the local directory. Nevertheless, all the other code in the recipe must be able to
run in the export folder inside the cache, where it has access only to the files exported (see
attribute :ref:`exports <exports_attribute>`) and to any other functionality
from a :ref:`python_requires <python_requires>`.
attribute :ref:`exports <exports_attribute>` and :ref:`conandata_yml`) and to any other functionality
from a :ref:`python_requires <python_requires>` package.

.. warning::

By default, in Conan v1.x the information after evaluating the attribute ``scm`` will be stored in the
*conanfile.py* file (the recipe will be modified when exported to the Conan cache) and any value will be
written in plain text (watch out about passwords).
However, you can activate the :ref:`scm_to_conandata<conan_conf>` config option, the *conanfile.py*
won't be modified (data is stored in a different file) and the fields ``username`` and ``password`` won't be
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Username and password are NEVER stored or they are not stored only if I set scm_to_conandata = True?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previous behavior is exactly the same, no changes, conanfile.py will be modified and the user/password will be stored in plain text.

New behavior, activated with scm_to_conandata=True won't store user/passw, won't modify the recipe file (breaking, opt-in)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, it is fine then 👍

stored, so these one will be computed each time the recipe is loaded.

.. note::

Expand Down
9 changes: 7 additions & 2 deletions reference/config_files/conan.conf.rst
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ The typical location of the **conan.conf** file is the directory ``~/.conan/``:
# conan_cmake_program = cmake # environment CONAN_CMAKE_PROGRAM (overrides the make program used in CMake.cmake_program)

# cmake_generator # environment CONAN_CMAKE_GENERATOR
# https://vtk.org/Wiki/CMake_Cross_Compiling
# cmake_generator_platform # environment CONAN_CMAKE_GENERATOR_PLATFORM
# http://www.vtk.org/Wiki/CMake_Cross_Compiling
# cmake_toolchain_file # environment CONAN_CMAKE_TOOLCHAIN_FILE
# cmake_system_name # environment CONAN_CMAKE_SYSTEM_NAME
# cmake_system_version # environment CONAN_CMAKE_SYSTEM_VERSION
Expand All @@ -59,6 +59,7 @@ The typical location of the **conan.conf** file is the directory ``~/.conan/``:
# temp_test_folder = True # environment CONAN_TEMP_TEST_FOLDER

# cacert_path # environment CONAN_CACERT_PATH
# scm_to_conandata # environment CONAN_SCM_TO_CONANDATA

[storage]
# This is the default path, but you can write your own. It must be an absolute path or a
Expand All @@ -79,7 +80,7 @@ The typical location of the **conan.conf** file is the directory ``~/.conan/``:
# You can skip the proxy for the matching (fnmatch) urls (comma-separated)
# no_proxy_match = *bintray.com*, https://myserver.*

[hooks] # environment CONAN_HOOKS
[hooks] # environment CONAN_HOOKS
attribute_checker

# Default settings now declared in the default profile
Expand Down Expand Up @@ -191,6 +192,10 @@ in the conan code base, allowing to debug the detected error.
The ``cacert_path`` variable lets the user specify a custom path to the *cacert.pem* file to use
in requests. You can also adjust this value using the environment variable ``CONAN_CACERT_PATH``.

The ``scm_to_conandata`` variable tells Conan to store the resolved information of the :ref:`SCM feature<scm_feature>` in the
:ref:`conandata.yml<conandata_yml>` file instead of modifying the recipe file itself. You can also adjust
this value using the environment variable ``CONAN_SCM_TO_CONANDATA``.

The ``skip_broken_symlinks_check`` variable (defaulted to ``False``) allows the existence broken symlinks while creating a package.

Storage
Expand Down
7 changes: 6 additions & 1 deletion reference/config_files/conandata.yml.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
conandata.yml
=============

This YAML file can be used to in the recipe to declare specific information to be used inside the recipe. This file is specific to each
This YAML file can be used to declare specific information to be used inside the recipe. This file is specific to each
recipe *conanfile.py* and it should be placed next to it. The file is automatically exported with the recipe (no need to add it to
:ref:`exports_attribute` attribute) and its content is loaded into the :ref:`conandata_attribute` attribute of the recipe.

Expand All @@ -24,3 +24,8 @@ For example:
patches: "0001-beast-fix-moved-from-executor.patch,bcp_namespace_issues.patch"
1.71.0:
patches: "bcp_namespace_issues.patch,boost_build_qcc_fix_debug_build_parameter.patch"


.. note::

The first level entry key ``.conan`` is reserved for Conan usage.