Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[question] can I create a dependency only lockfile for a package? #9400

Closed
1 task done
planetmarshall opened this issue Aug 8, 2021 · 3 comments
Closed
1 task done
Assignees

Comments

@planetmarshall
Copy link
Contributor

If I have a package with a conanfile.txt, then the lockfile created contains the dependency graph of the dependencies, as I would expect.

If however I am working on a package that has a conanfile.py, then the lockfile contains the dependency graph in addition to the package itself.

It would be useful in the latter case to only create a lockfile for the dependencies - specifically, a dependency-only lockfile is useful to use as a hash key for CI caching mechanisms. I can't use a lockfile which contains the package itself for this purpose as the version information for the package under development may change on each commit, which means the cache key is never the same.

@planetmarshall
Copy link
Contributor Author

A bit of context for this - I'm trying to make use of the caching feature available on Github Workflows and Azure Pipelines. At the moment I have something like this:

- name: create lockfile
  run: | 
    conan lock create conanfile.py \
      --profile profile \
      -o some_option=value \
      --build missing

- name: cache conan dependencies
  uses: actions/cache@v2
  with:
    path: ~/.conan/data
    key: conan-${{ hashFiles('conan.lock') }}
    restore-keys: conan-profile

- name: install conan dependencies
  run: |
      conan install . \
        --install-folder build \
        --lockfile conan.lock \
        --build missing

That seems to work pretty well provided the version number of the package's conanfile.py doesn't change. However, we have some logic which calculates the version number based on git history, and so it changes on each commit.

Where I have my own artifactory instance, I can work around this by having a fixed lockfile in source control which is used to cache and install dependencies, and then from that lockfile I derive a second one to actually build the package, something like:

# dependencies.lock is committed to source control, but won't have the correct version number for the package
$ conan lock create conanfile.py . --lockfile dependencies.lock --lockfile-out package.lock
$ conan create . --lockfile package.lock

However, that only works when I have an artifactory instance with all the packages I need pre-built. If there are any binaries missing, as there might be when using CCI, this won't work because I'd need to know in advance which packages are missing.

I think what I want is the ability to create a lockfile for all the package's dependencies (including options and settings), but also allows me to build any of those packages that might be missing.

@planetmarshall
Copy link
Contributor Author

Resolved by creating a lockfile via conan install rather than conan create.

@memsharded
Copy link
Member

Great, thanks for following up, sorry that we didn't respond on time, this issue fell through the cracks (probably issue being filed in August, when many of us typically take our vacation..)

I guess you might be interested in the new lockfiles for 2.0, I'd recommend having a look (conan-io/tribe#34) and trying them in the released 2.0.beta. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants