Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kernelci/build.py: Add apply_patch_mbox method #2041

Closed
wants to merge 1 commit into from
Closed

kernelci/build.py: Add apply_patch_mbox method #2041

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions kernelci/build.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (C) 2018, 2019, 2020, 2021 Collabora Limited

Check warning on line 1 in kernelci/build.py

View workflow job for this annotation

GitHub Actions / Lint 

Too many lines in module (1574/1000)

Check warning on line 1 in kernelci/build.py

View workflow job for this annotation

GitHub Actions / Lint 

Missing module docstring
# Author: Guillaume Tucker <guillaume.tucker@collabora.com>
#
# This module is free software; you can redistribute it and/or modify it under
Expand All @@ -24,6 +24,7 @@
import re
import shutil
import tarfile
import tempfile
import time
import urllib.parse

Expand All @@ -41,7 +42,7 @@
/raw/master/{branch}/{config}"

CROS_CONFIG_URL = \
"https://chromium.googlesource.com/chromiumos/third_party/kernel/+archive/refs/heads/{branch}/chromeos/config.tar.gz" # noqa

Check warning on line 45 in kernelci/build.py

View workflow job for this annotation

GitHub Actions / Lint 

Line too long (129/100)

# Hard-coded make targets for each CPU architecture
MAKE_TARGETS = {
Expand Down Expand Up @@ -76,7 +77,7 @@
The returned value is the git SHA of the current head of the branch
associated with the build config, or None if an error occurred.
"""
cmd = "git ls-remote {url} refs/heads/{branch}".format(

Check warning on line 80 in kernelci/build.py

View workflow job for this annotation

GitHub Actions / Lint 

Formatting a regular string which could be a f-string
url=config.tree.url, branch=config.branch)
head = shell_cmd(cmd)
if not head:
Expand All @@ -100,7 +101,7 @@


def _fetch_tags(path, url=TORVALDS_GIT_URL):
shell_cmd("""

Check warning on line 104 in kernelci/build.py

View workflow job for this annotation

GitHub Actions / Lint 

Formatting a regular string which could be a f-string
set -e
cd {path}
git fetch --tags {url}
Expand All @@ -124,7 +125,7 @@
_update_remote(config, path)


def clone_git(url, path, branch, shallow=True):

Check warning on line 128 in kernelci/build.py

View workflow job for this annotation

GitHub Actions / Lint 

Unused argument 'shallow'
"""Lightweight git repo clone
*url* git repo url
*path* destination directory
Expand All @@ -132,7 +133,7 @@
"""
if not os.path.exists(path):
shell_cmd(f"git clone {url} {path}")
shell_cmd("""

Check warning on line 136 in kernelci/build.py

View workflow job for this annotation

GitHub Actions / Lint 

Formatting a regular string which could be a f-string
set -e
cd {path}
git reset --hard
Expand All @@ -150,8 +151,8 @@
*ref* is the path to a reference repo, typically a mirror
"""
if not os.path.exists(path):
ref_opt = '--reference={ref}'.format(ref=ref) if ref else ''

Check warning on line 154 in kernelci/build.py

View workflow job for this annotation

GitHub Actions / Lint 

Formatting a regular string which could be a f-string
shell_cmd("git clone {ref} -o {remote} {url} {path}".format(

Check warning on line 155 in kernelci/build.py

View workflow job for this annotation

GitHub Actions / Lint 

Formatting a regular string which could be a f-string
ref=ref_opt, remote=config.tree.name,
url=config.tree.url, path=path))

Expand Down Expand Up @@ -321,6 +322,37 @@
return False


def apply_patch_mbox(
kdir,
mbox_url,
git_username="kernelci-tsc",
git_email="kernelci-tsc@groups.io"
):
"""Download patch mbox from URL and apply with 3-way merge

*kdir* is the path to a kernel source directory
*mbox_url* is the URL to patch mbox content
*git_username* is the username used to apply the patch
*git_email* is the email used to apply the patch
"""
with tempfile.NamedTemporaryFile(prefix="kernel-patch-") as tmp_f:
yurinnick marked this conversation as resolved.
Show resolved Hide resolved
if not _download_file(mbox_url, tmp_f.name):
raise FileNotFoundError(f"Error downloading patch mbox {mbox_url}")

shell_cmd("""
set -e
cd {kdir}
git config user.name "{git_username}"
git config user.email "{git_email}"
Comment on lines +345 to +346
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No I mean, I think we should actually remove these lines. Say, if you're doing this over your working kernel source tree and you don't specify command line arguments with your current user name and email then it will replace it. And then next time you make a commit in the kernel source tree it'll be with the kernelci-tsc@groups.io email.

So it should be left to the user to manage this outside of kci_build.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean it should be part of KernelCI config files or something? Sorry, I am not quite following what do you mean. Git won't allow to apply patches without name and email being set though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be part of the user's Git config, yes. Or we could maybe have a separate command to set the user name and email explicitly if some CI systems need to do that, as a handy wrapper.

Git won't allow to apply patches without name and email being set though.

Yes and that's fine. If there's no user name and email configured then git will fail and the issue will be on the user to solve it. Just like running any other invalid command (say, if kdir is not a Git repo etc.).

git am --3way {mbox_file}
""".format(
kdir=kdir,
mbox_file=tmp_f.name,
git_username=git_username,
git_email=git_email
))


def pull_tarball(kdir, url, dest_filename, retries, delete):
if os.path.exists(kdir):
shutil.rmtree(kdir)
Expand Down Expand Up @@ -1151,7 +1183,7 @@
res = self._make(target, jopt, verbose, opts)

if res and kci_frag_name:
# ToDo: treat kernelci.config as an implementation detail and list

Check warning on line 1186 in kernelci/build.py

View workflow job for this annotation

GitHub Actions / Lint 

ToDo: treat kernelci.config as an implementation detail and list
# the actual input config fragment files here instead
bmeta['kernel']['fragments'] = [kci_frag_name]
res = self._merge_config(kci_frag_name, verbose)
Expand Down