Recommended mechanism to copy/archive a repository

[BrianSipos]

I've been enjoying the repeatability that conan brings to building C/C++ libraries, and have run into a gap in my understanding of package/repository management. In YUM/RPM world, a YUM repository is a filesystem tree which can be tar/zip bundled and copied or archived, which is very convenient behavior.

In conan, when recipes/packages are cached locally they are distributed among user home directory ".conan" and short-path directory "C:.conan" which makes it difficult to make a full copy of a local cache. Is there a canonical or recommended way to zip up a repository or local cache in a redistributable way?

I use Artifactory to manage local conan repositories and have considered to use the Artifactory export form for conan repository redistribution, but it seems like this ties the repository tightly to Artifactory which I would like to avoid if possible.

[memsharded]

Hi @BSipos-RKF

The local cache, as it is named "cache" is not intended to be zipped and redistributed. It might contain temporary files (e.g. lock-files), it might contain absolute paths that are specific to that instance: "C:/Users/username/.conan/data/boost/1.68....". It will also contain temporary build objects, than in most cases will be extremely heavy, and the builds will not be relocatable anyway...

You probably want to make sure to clean the build and source folders, to clean the locks, and in general absolute paths won't be a problem in package artifacts, so you might try to zip both the .conan/data folder and the C:/.conan folder, and assuming you will put the later in the same location in the final system, it might work. But I wouldn't recommend this in production, but rather using Artifactory sounds a more robust approach.

[BrianSipos]

I was trying to look into the documentation for conan_server to see if there is a canonical filesystem layout and I don't see any explicit documentation. I would prefer to align my Artifactory configuration with the same file layout that conan_server uses.

It appears that the Artifactory conan-default layout is [org]/[module]/[baseRev]/[channel<[^/]+>][remainder<(?:.*)>].[ext] which will place conan package "name/version@user/channel" into path "user/name/version/channel". Is this the same layout that conan_server uses and/or requires?

[pwuertz]

Was looking for something like this. So if I want to transfer a built package from system A to system B, I need to set up a Conan server that is accessible by both systems? I.e. there is no support for exporting / importing / backup built packages as some sort of archive like, say a python wheel?

[memsharded]

Was looking for something like this. So if I want to transfer a built package from system A to system B, I need to set up a Conan server that is accessible by both systems? I.e. there is no support for exporting / importing / backup built packages as some sort of archive like, say a python wheel?

No, at the moment there is not. We are considering implementing something after 2.0 (2.0 is already in alpha), but at the moment you need a server to upload the packages.

[memsharded]

Conan 2.0 is out.

Conan 2.0.3 will include (hidden initially, for team testing) #13461, which includes the "backup-sources" to backup interned download()ed sources. Still not what is being described here, but I thought it would be relevant.

At the moment, we are not considering to make the local cache redistributable, because it is very much designed and intended as that, a cache. The very recommended way and the source of truth to archive repositories are servers (this is also true for many other package types like maven, docker, npm, etc).

Conan 2.x will probably provide some support for doing that at the package level, which might be extended by users to do for the whole cache, but it sounds quite overkill, and still the server usage is recommended (at the end of the day, transferring from system A to system B many times uses an intermediate server anyway). I am labeling this as 2.X, just in case.

[memsharded]

Implemented in #14923, for next 2.0.14 release.