Recent git update has broken builds using docker images

[charris]

See https://github.blog/2022-04-12-git-security-vulnerability-announced/. The effect of this change is that git will no longer recognize all repos as belonging to the git owner, this breaks versioneer and git describe, so that projects that depend on them for versioning no longer build. See MacPython/scipy-wheels#167 for the problem, along with some links to NumPy fixes that worked for the NumPy CI.

[mattip]

I think the proper fix is to add safe.directory like in the NumPy fixes. See pypa/manylinux#1309

[tylerjereddy]

I haven't had much luck here so far--please do ping me if you see another project do something useful in this regard, etc.

[mattip]

I guess multibuild could add git config --global --add safe.directory $1 to the gh_clone function but is that in common use?

[matthew-brett]

@tylerjereddy - did you try something like Matti's suggestion?

function gh_clone {
    git clone https://github.com/$1
    git config --global --add safe.directory $1
}

[tylerjereddy]

Ah, I did something similar, but it was elsewhere in the control flow. There's also a suggestion from Henry over there: pypa/manylinux#1309 (comment)

Maybe I will try these ideas later today, thanks!

[mattip]

I see another git step that is actually used by scipy wheels in clean_code, would it be appropriate to add the git config here?

multibuild/common_utils.sh

Lines 282 to 298 in e0b6e4f

	function clean_code {
	local repo_dir=${1:-$REPO_DIR}
	local build_commit=${2:-$BUILD_COMMIT}
	[ -z "$repo_dir" ] && echo "repo_dir not defined" && exit 1
	[ -z "$build_commit" ] && echo "build_commit not defined" && exit 1
	# The package $repo_dir may be a submodule. git submodules do not
	# have a .git directory. If $repo_dir is copied around, tools like
	# Versioneer which require that it be a git repository are unable
	# to determine the version. Give submodule proper git directory
	fill_submodule "$repo_dir"
	(cd $repo_dir \
	&& git fetch origin \
	&& git checkout $build_commit \
	&& git clean -fxd \
	&& git reset --hard \
	&& git submodule update --init --recursive)
	}

[charris]

That looks like a likely spot. git config needs to be added before switching to the build directory, and issuing any other git commands.

[charris]

Note that in one case I needed to use --system instead of --global because a root directory wasn't found, that probably isn't a problem here.

[tylerjereddy]

I see the same thing I saw with my previous attempts so far:

2022-05-02T01:50:22.8996044Z fatal: unsafe repository ('/io/scipy' is owned by someone else)
2022-05-02T01:50:22.8997327Z To add an exception for this directory, call:
2022-05-02T01:50:22.8997722Z 
2022-05-02T01:50:22.8998420Z 	git config --global --add safe.directory /io/scipy
2022-05-02T01:50:22.9017913Z fatal: unsafe repository ('/io/scipy' is owned by someone else)
2022-05-02T01:50:22.9018869Z To add an exception for this directory, call:
2022-05-02T01:50:22.9019256Z 
2022-05-02T01:50:22.9019942Z 	git config --global --add safe.directory /io/scipy

[tylerjereddy]

(and then the version number looks weird again)

[matthew-brett]

I guess you have to apply this command also in the scipy submodule.

[tylerjereddy]

More diagnostic info/local reproducer: MacPython/scipy-wheels#167 (comment)

[tylerjereddy]

Henry's scoping of the git rev-parse command seems to be the key, the git config .. command was useless for me both in CI and locally.

[charris]

Hmm. This doesn't directly apply for NumPy, looks like I need to start working on that.

[charris]

Found a fix using the config.sh file, see MacPython/numpy-wheels#160. NumPy uses versioneer.

[has2k1]

What worked for me on github actions was adding

if [ $(uname) == "Linux" ]; then
    IS_LINUX=1
    ! git config --global --add safe.directory "*"
fi

to the top of config.sh. I could not figure out the exact directory to make safe.

[mattip]

Closing. I think end users found the work-arounds they needed. Please reopen or open a new issue if I am mistaken.