Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cargo: Allow filtering packages by name #325

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 27 additions & 4 deletions cargo/flatpak-cargo-generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,18 @@ async def get_package_sources(package, cargo_lock, git_repos):
return (crate_sources, {'crates-io': {'replace-with': VENDORED_SOURCES}})


async def generate_sources(cargo_lock, git_tarballs=False):
def packages_dependencies(names, all_packages):
for name in names or ():
# Some are like "time 0.3.15"
name = name.split(' ')[0]
yield name
package = all_packages[name]
deps = package.get('dependencies')
for dep in packages_dependencies(deps, all_packages):
yield dep
Comment on lines +297 to +298
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for dep in packages_dependencies(deps, all_packages):
yield dep
yield from packages_dependencies(deps, all_packages)



async def generate_sources(cargo_lock, packages=None, git_tarballs=False):
# {
# "git-repo-url": {
# "lock": asyncio.Lock(),
Expand All @@ -305,12 +316,22 @@ async def generate_sources(cargo_lock, git_tarballs=False):
VENDORED_SOURCES: {'directory': f'{CARGO_CRATES}'},
}

pkg_coros = [get_package_sources(p, cargo_lock, git_repos) for p in cargo_lock['package']]
if packages:
packages = packages.split(',')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better do parsing earlier in the flow, ideally in main(), and pass packages here as a list of strings.

logging.debug('Considering only packages: %s', packages)
all_packages = {p['name']: p for p in cargo_lock['package']}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The all_packages dict feels out of place here, since it's only used within packages_dependencies() scope. Better pass the whole cargo_lock to packages_dependencies(), and there get whatever information is needed from it.

packages = list(packages_dependencies(packages, all_packages))
else:
logging.debug('Considering all packages')
packages = list([p['name'] for p in cargo_lock['package']])

pkg_coros = [get_package_sources(p, cargo_lock, git_repos)
for p in cargo_lock['package']
if p['name'] in packages]
for pkg in await asyncio.gather(*pkg_coros):
if pkg is None:
continue
else:
pkg_sources, cargo_vendored_entry = pkg
pkg_sources, cargo_vendored_entry = pkg
Comment on lines -312 to +334
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: please make unrelated changes in separate commits.

package_sources.extend(pkg_sources)
cargo_vendored_sources.update(cargo_vendored_entry)

Expand Down Expand Up @@ -338,6 +359,7 @@ async def generate_sources(cargo_lock, git_tarballs=False):
def main():
parser = argparse.ArgumentParser()
parser.add_argument('cargo_lock', help='Path to the Cargo.lock file')
parser.add_argument('-p', '--packages', help='Comma-separated packages in Cargo.lock to build')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This help text is a bit misleading, since we don't control what Cargo builds. Maybe "...list of package name to fetch" would be better?

parser.add_argument('-o', '--output', required=False, help='Where to write generated sources')
parser.add_argument('-t', '--git-tarballs', action='store_true', help='Download git repos as tarballs')
parser.add_argument('-d', '--debug', action='store_true')
Expand All @@ -353,6 +375,7 @@ def main():
logging.basicConfig(level=loglevel)

generated_sources = asyncio.run(generate_sources(load_toml(args.cargo_lock),
packages=args.packages,
git_tarballs=args.git_tarballs))
with open(outfile, 'w') as out:
json.dump(generated_sources, out, indent=4, sort_keys=False)
Expand Down