[question] Availability of self.export_folder

[madebr]

Hello,

I am in the process of automatically generating dependency information for boost.
See conan-io/conan-center-index#1925 (comment) for the progress.

Using export, I copy the dependency information to the export folder. This has to be copied to the export folder because the information needs to be available in package_info, where no source_folder or build_folder exists.

But self.export_folder is unavailable in package_info.

AttributeError: 'BoostConan' object has no attribute 'export_folder'

So my question is: How do I access the export folder in package_info?

• I've read the CONTRIBUTING guide.

[memsharded]

Hi @madebr

There is no access to the export folder in the package_info() method. If you need something from there, you need to package() it in the final package. The idea is that packages should be able to be consumed with what is inside the "package" folder only, nothing else. Whatever is in the other folders, the conanfile.py, etc is necessary to resolve the package folder, but after that, consumers, and for example, generators, should be only accessing package folders, but not export ones.

[madebr]

Thanks, it seemed useful to have this dependency information saved in one place.
But this will also work.

[madebr]

How can I use the generated file to check the options in configure?
export_sources or source haven't run yet.

[memsharded]

Not sure what you mean. Many times when a package is being consumed, there is no export_sources or sources at all, this is why things need to be packaged to be accessed by package_info().

[madebr]

Boost is a huge project and has a complex dependency tree.
The recipe at cci supports multiple versions.
That's why I generate the dependency tree for each verson,
and want to include that in the conanfile.

e.g. boost has wave and serialization options. wave depends on serialization.
Now I want to check in configure that when wave is enabled, but serialization is not, that a ConanInvalidConfiguration is thrown.

[memsharded]

I see, yes, it might make sense to raise a ConanInvalidConfiguration if the inputs are incorrect, at configure() time. But that is completely possible right now, isn't it. What am I missing? I am sorry, now I don't see how this is related to the package_info() accessing the export folder.

[madebr]

I want also access to the information of the yml in configure, but can't because neither self.export_folder and self.souce_folder is available.

[memsharded]

The conandata.yml? but that is automatically loaded as self.conan_data attribute, isn't it? No need to access the file directly?

[madebr]

I'm generating a yml with the dependency information.
e.g.

Dependency information of boost 1.73.0

buildables:
- atomic
- chrono
- container
- context
- contract
- coroutine
- date_time
- exception
- fiber
- filesystem
- graph
- graph_parallel
- iostreams
- locale
- log
- math
- mpi
- nowide
- program_options
- python
- random
- regex
- serialization
- signals
- stacktrace
- system
- test
- thread
- timer
- type_erasure
- wave
dependencies:
  atomic:
  - assert
  - config
  - static_assert
  - type_traits
  chrono:
  - assert
  - config
  - core
  - integer
  - move
  - mpl
  - predef
  - ratio
  - static_assert
  - system
  - throw_exception
  - type_traits
  - typeof
  - utility
  - winapi
  container:
  - assert
  - config
  - container_hash
  - core
  - intrusive
  - move
  - static_assert
  - type_traits
  context:
  - assert
  - config
  - pool
  - predef
  - smart_ptr
  - thread
  contract:
  - any
  - assert
  - config
  - core
  - exception
  - function
  - function_types
  - mpl
  - optional
  - preprocessor
  - smart_ptr
  - static_assert
  - thread
  - type_traits
  - typeof
  - utility
  coroutine:
  - assert
  - config
  - context
  - core
  - exception
  - move
  - range
  - system
  - thread
  - throw_exception
  - type_traits
  - utility
  date_time:
  - algorithm
  - assert
  - config
  - core
  - io
  - lexical_cast
  - range
  - serialization
  - smart_ptr
  - static_assert
  - throw_exception
  - tokenizer
  - type_traits
  - utility
  - winapi
  exception:
  - assert
  - config
  - core
  - smart_ptr
  - throw_exception
  - tuple
  - type_traits
  fiber:
  - algorithm
  - assert
  - config
  - context
  - core
  - filesystem
  - format
  - intrusive
  - predef
  - smart_ptr
  filesystem:
  - assert
  - config
  - container_hash
  - core
  - detail
  - io
  - iterator
  - predef
  - smart_ptr
  - system
  - type_traits
  - winapi
  graph:
  - algorithm
  - any
  - array
  - assert
  - bimap
  - bind
  - concept_check
  - config
  - container_hash
  - conversion
  - core
  - detail
  - foreach
  - function
  - integer
  - iterator
  - lexical_cast
  - math
  - move
  - mpl
  - multi_index
  - optional
  - parameter
  - preprocessor
  - property_map
  - property_tree
  - random
  - range
  - regex
  - serialization
  - smart_ptr
  - spirit
  - static_assert
  - throw_exception
  - tti
  - tuple
  - type_traits
  - typeof
  - unordered
  - utility
  - xpressive
  graph_parallel:
  - assert
  - concept_check
  - config
  - container_hash
  - core
  - detail
  - dynamic_bitset
  - filesystem
  - foreach
  - function
  - graph
  - iterator
  - lexical_cast
  - mpi
  - mpl
  - optional
  - property_map
  - random
  - serialization
  - smart_ptr
  - static_assert
  - tuple
  - type_traits
  - variant
  iostreams:
  - assert
  - bind
  - config
  - core
  - detail
  - function
  - integer
  - iterator
  - mpl
  - preprocessor
  - random
  - range
  - regex
  - smart_ptr
  - static_assert
  - throw_exception
  - type_traits
  - utility
  locale:
  - assert
  - config
  - function
  - iterator
  - smart_ptr
  - static_assert
  - thread
  - type_traits
  - unordered
  log:
  - align
  - array
  - asio
  - assert
  - atomic
  - bind
  - config
  - container
  - core
  - date_time
  - exception
  - filesystem
  - function_types
  - fusion
  - interprocess
  - intrusive
  - io
  - iterator
  - lexical_cast
  - locale
  - move
  - mpl
  - optional
  - parameter
  - phoenix
  - predef
  - preprocessor
  - property_tree
  - proto
  - random
  - range
  - regex
  - smart_ptr
  - spirit
  - static_assert
  - system
  - thread
  - throw_exception
  - type_index
  - type_traits
  - utility
  - winapi
  - xpressive
  math:
  - array
  - assert
  - atomic
  - concept_check
  - config
  - core
  - detail
  - fusion
  - integer
  - lambda
  - lexical_cast
  - mpl
  - predef
  - range
  - smart_ptr
  - static_assert
  - throw_exception
  - tuple
  - type_traits
  mpi:
  - assert
  - config
  - core
  - foreach
  - function
  - graph
  - integer
  - iterator
  - lexical_cast
  - mpl
  - optional
  - python
  - serialization
  - smart_ptr
  - static_assert
  - throw_exception
  - type_traits
  - utility
  nowide:
  - config
  - filesystem
  - smart_ptr
  - static_assert
  program_options:
  - any
  - bind
  - config
  - core
  - detail
  - function
  - iterator
  - lexical_cast
  - smart_ptr
  - static_assert
  - throw_exception
  - tokenizer
  - type_traits
  python:
  - bind
  - config
  - conversion
  - core
  - detail
  - foreach
  - function
  - graph
  - integer
  - iterator
  - lexical_cast
  - mpl
  - preprocessor
  - property_map
  - smart_ptr
  - static_assert
  - tuple
  - type_traits
  - utility
  random:
  - assert
  - config
  - core
  - dynamic_bitset
  - integer
  - io
  - math
  - mpl
  - multiprecision
  - range
  - static_assert
  - system
  - throw_exception
  - type_traits
  - utility
  regex:
  - assert
  - concept_check
  - config
  - container_hash
  - core
  - integer
  - iterator
  - mpl
  - predef
  - smart_ptr
  - static_assert
  - throw_exception
  - type_traits
  serialization:
  - array
  - assert
  - config
  - core
  - detail
  - function
  - integer
  - io
  - iterator
  - move
  - mpl
  - optional
  - predef
  - preprocessor
  - smart_ptr
  - spirit
  - static_assert
  - type_traits
  - unordered
  - utility
  - variant
  signals:
  - any
  - config
  - core
  - function
  - iterator
  - mpl
  - optional
  - smart_ptr
  - type_traits
  - utility
  stacktrace:
  - array
  - config
  - container_hash
  - core
  - predef
  - static_assert
  - type_traits
  - winapi
  system:
  - config
  - winapi
  test:
  - algorithm
  - assert
  - bind
  - config
  - core
  - detail
  - exception
  - function
  - io
  - iterator
  - mpl
  - optional
  - preprocessor
  - smart_ptr
  - static_assert
  - type_traits
  - utility
  thread:
  - algorithm
  - assert
  - atomic
  - bind
  - chrono
  - concept_check
  - config
  - container
  - container_hash
  - core
  - date_time
  - exception
  - function
  - intrusive
  - io
  - iterator
  - lexical_cast
  - move
  - optional
  - predef
  - preprocessor
  - smart_ptr
  - static_assert
  - system
  - throw_exception
  - tuple
  - type_traits
  - utility
  - winapi
  timer:
  - chrono
  - config
  - core
  - io
  - predef
  - system
  - throw_exception
  type_erasure:
  - assert
  - config
  - core
  - fusion
  - iterator
  - mpl
  - preprocessor
  - smart_ptr
  - thread
  - throw_exception
  - type_traits
  - typeof
  - vmd
  wave:
  - assert
  - concept_check
  - config
  - core
  - filesystem
  - iterator
  - lexical_cast
  - mpl
  - multi_index
  - pool
  - preprocessor
  - serialization
  - smart_ptr
  - spirit
  - static_assert
  - throw_exception
  - type_traits
libs:
  atomic:
  - boost_atomic
  chrono:
  - boost_chrono
  container:
  - boost_container
  context:
  - boost_context
  contract:
  - boost_contract
  coroutine:
  - boost_coroutine
  date_time:
  - boost_date_time
  exception:
  - boost_exception
  fiber:
  - boost_fiber
  filesystem:
  - boost_filesystem
  graph:
  - boost_graph
  graph_parallel:
  - boost_graph_parallel
  iostreams:
  - boost_iostreams
  locale:
  - boost_locale
  log:
  - boost_log
  - boost_log_setup
  math:
  - boost_math_c99f
  - boost_math_c99l
  - boost_math_c99
  - boost_math_tr1f
  - boost_math_tr1l
  - boost_math_tr1
  mpi:
  - boost_mpi
  nowide:
  - boost_nowide
  program_options:
  - boost_program_options
  python:
  - boost_python
  random:
  - boost_random
  regex:
  - boost_regex
  serialization:
  - boost_serialization
  - boost_wserialization
  signals:
  - boost_signals
  stacktrace:
  - boost_stacktrace
  - boost_stacktrace_noop
  - boost_stacktrace_backtrace
  - boost_stacktrace_addr2line
  - boost_stacktrace_basic
  - boost_stacktrace_windbg
  - boost_stacktrace_windbg_cached
  system:
  - boost_system
  test:
  - boost_unit_test_framework
  - boost_prg_exec_monitor
  - boost_test_exec_monitor
  thread:
  - boost_thread
  timer:
  - boost_timer
  type_erasure:
  - boost_type_erasure
  wave:
  - boost_wave
version: 1.73.0

[memsharded]

Ah, another file. Now I understand the issue, many thanks.

Yes, at the moment the configure() doesn't have definition of the export folder, so it cannot access files there manually. While we consider what would be the way for this use case, that is currently not possible, I would like to ask: have you considered adding that information to the conandata.yml instead? That file will be automatically exported first, and loaded later, so it could be very convenient. I am trying to understand and learn if this could be the recommended way for this use case or not.

[madebr]

It would work, but the file would get huge for multiple boost versions.
Also, the rules of conan-center-index currently disallow keys, except for those related to sources and patches.

[madebr]

I've used export_sources + package now.
The downside of this is that when I change information in the dependency information, conan export will not apply the change.

[jgsogo]

I'm answering here, although it might apply to conan-io/conan-center-index#1925 or conan-io/conan-center-index#2097

IMO this information should be stored in the conandata.yml file, it is information needed by the recipe and I think it is a cleaner approach to use conandata.yml than to read a file stored together with the recipe.

That information is needed in the package_info function, and it will be available if stored in conandata.yml. The problem is where that information comes from: it can be hardcoded or dynamically generated using boostdep (and some postprocessing).

• If the information about dependencies is hardcoded somewhere, we can use the export() function to read that file (the one for the version we are packaging) and copy that information in the final conandata.yml. Final package in the server will contain only the information related to the specific version we are packaging (like we do with scm_to_conandata.
  I know we need to modify the conan-center hook to allow this extra info, not a big deal if this is the way to go.

• If we want to compute the information dynamically using boostdep, then we need to build_require("boost") because the binary boostdep needs to run in the build machine, we cannot use the binary generated by the boost compilation we are running... at this moment, and for this usage, I wouldn't add this circular dependency.

Then the hardcoded way: copying contents from a file in the repository to the conandata.yml in the export() function, no need to export the file with the hardcoded information.

wdyt about it? Is there anything in Conan itself that prevents us from doing something like this?

[madebr]

Nope, there is nothing in conan preventing this.
Just the rules on CCI that are currently too tight.

I'm left wondering though, what is the purpose of export and exports if one can only access files in the export function?
As I see it now, it is only useful for exporting licenses.

[jgsogo]

export/exports is to gather files/information that are needed by the conanfile.py file itself, like conandata.yml, the license, metadata,... and for the boost package, for example, it is a good place to read the file that contains the dependencies and store only the information matching the version in the conandata.yml file:

(I haven't run this, I'm not sure about working directories)

class BoostRecipe(ConanFile):
    def export(self):
        # Read the file for the version we are packaging
        data = load("dependencies-{}.yml".format(self.version))
        # Add that information to the conandata.yml
        conandata = load("conandata.yml")
        conandata["dependencies"] = data
        save("{export_folder}/conandata.yml", conandata)

    def package_info(self):
        dependencies = self.conan_data["dependencies"]

Of course, this needs to be considered by the CCI hook, it won't work without modifying it.