-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Symlink failures when installing azure-cli brew package installs dependency on Python 3.11 causes regression and build breaks. #9471
Comments
on a side note, the build agent images have python 3.11 and 3.12 preinstalled, so why is brew even trying to install them as a package dependency, is there a minor 3.11.xx build version different for these packages that brew is trying to match? Or are you using something other than brew to install these, so brew just does not know they are already installed? |
If anyone else is running into this, I was able to work around this initial failure by uninstalling and then cleaning up the following packages and dirty folders, and then letting brew reinstall these packages when it detected them as dependencies.
|
Though the above does not solve all the regressions, as "brew install doxygen" is still failing later in our builds, with this strange error. Warning: Treating doxygen as a formula. For the cask, use homebrew/cask/doxygen ##[error]Bash exited with code '134'. |
Hey @BrianMouncer! It's a |
Is there a workaround that you know of? I'm not the one calling "brew install pythonxxx", brew is doing it as a dependency of another package, and even when I try to run the command early on my own, running "brew link --overwrite python@3.11" does not work, because those folders exist and can't be linked overtop of. I was able to eventually build a list of folders to delete, and then the brew python install/link would works. @erik-bershel are you aware of this issue too, and a possible fix? Warning: Treating doxygen as a formula. For the cask, use homebrew/cask/doxygen ##[error]Bash exited with code '134'. |
Promoting Homebrew's python@3.12 to the default python3 breaks symbolic links on macOS x86_64. This change adds a workaround for that issue. Also see: actions/runner-images#9471 etc.
Promoting Homebrew's python@3.12 to the default python3 breaks symbolic links on macOS x86_64. This change adds a workaround for that issue. Also see: actions/runner-images#9471 etc.
Fix recurring [macos CI issue](actions/runner-images#9471) (see https://github.com/model-checking/kani/actions/runs/8234799856/job/22521976090?pr=3065#step:3:202 for an example of a failing run) and do a `cargo update`. This subsumes #3065 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.
In my duplicate issue, this was caused by the However, the
|
Promoting Homebrew's python@3.12 to the default python3 breaks symbolic links on macOS x86_64. This change adds a workaround for that issue. Also see: actions/runner-images#9471 etc.
Promoting Homebrew's python@3.12 to the default python3 breaks symbolic links on macOS x86_64. This change adds a workaround for that issue. Also see: actions/runner-images#9471 etc.
@erik-bershel maybe an easy solution would be to allow us to pick different macOS runner images and provide at least two options: one with batteries included (the current image, make it default) and one barebones image (with only homebrew installed). This way we can avoid this issue completely. While it's useful to have Python already there for some users, for others it will be a source of problems. So why not provide choice? |
Promoting Homebrew's python@3.12 to the default python3 breaks symbolic links on macOS x86_64. This change adds a workaround for that issue. Also see: actions/runner-images#9471 etc. Github-Pull: bitcoin#29610 Rebased-From: ae5f720
Promoting Homebrew's python@3.12 to the default python3 breaks symbolic links on macOS x86_64. This change adds a workaround for that issue. Also see: actions/runner-images#9471 etc. Github-Pull: bitcoin#29610 Rebased-From: ae5f720
Homebrew and system installed python have an unfortunate interaction from time to time. Most recent github statement on this: actions/runner-images#9471 (comment) We already had a workaround in place from c6129fc, but it is not working for the current round of the bug. I think reordering should be the fix. Previously this CI issue must have triggered later in our build process (currently unclear to me how; maybe the brew calls in the prior step only recently triggered a dependency upgrade; right now the culprit is libpng). This commit should ensure it is always triggered before any brew calls.
Promoting Homebrew's python@3.12 to the default python3 breaks symbolic links on macOS x86_64. This change adds a workaround for that issue. Also see: actions/runner-images#9471 etc. Github-Pull: bitcoin#29610 Rebased-From: ae5f720
Promoting Homebrew's python@3.12 to the default python3 breaks symbolic links on macOS x86_64. This change adds a workaround for that issue. Also see: actions/runner-images#9471 etc. Github-Pull: bitcoin#29610 Rebased-From: acc06bc
Promoting Homebrew's python@3.12 to the default python3 breaks symbolic links on macOS x86_64. This change adds a workaround for that issue. Also see: actions/runner-images#9471 etc. Github-Pull: bitcoin#29610 Rebased-From: ae5f720
Promoting Homebrew's python@3.12 to the default python3 breaks symbolic links on macOS x86_64. This change adds a workaround for that issue. Also see: actions/runner-images#9471 etc.
Promoting Homebrew's python@3.12 to the default python3 breaks symbolic links on macOS x86_64. This change adds a workaround for that issue. Also see: actions/runner-images#9471 etc.
Description
we install doxygen build depencancies with this brew command.
brew install pkg-config coreutils bash ninja jq gnu-getopt findutils gawk
This is suddenly failing, causing build breaks related to azure-cli and python 3.11 installs.
Platforms affected
Runner images affected
Image version and build link
Starting: Initialize job
Agent name: 'Azure Pipelines 143'
Agent machine name: 'Mac-1709791267116'
Current agent version: '3.236.1'
Operating System
macOS
13.6.4
22G513
Runner Image
Image: macos-13
Version: 20240219.1
Included Software: https://github.com/actions/runner-images/blob/macos-13/20240219.1/images/macos/macos-13-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/macos-13%2F20240219.1
Runner Image Provisioner
Current image version: '20240219.1'
Agent running as: 'runner'
Prepare build directory.
Set build variables.
Download all required tasks.
Downloading task: CmdLine (2.231.1)
Downloading task: 1ESHostedPoolValidation (1.0.27164358)
Downloading task: nuget-security-analysis (0.2.196)
Downloading task: CodeQL3000Init (0.1.291)
Downloading task: Bash (3.236.1)
Downloading task: PublishBuildArtifacts (1.231.1)
Downloading task: ComponentGovernanceComponentDetection (0.2420228.1)
Downloading task: CodeQL3000Finalize (0.1.291)
Checking job knob settings.
Knob: DockerActionRetries = true Source: $(VSTSAGENT_DOCKER_ACTION_RETRIES)
Knob: AgentToolsDirectory = /Users/runner/hostedtoolcache Source: ${AGENT_TOOLSDIRECTORY}
Knob: AgentEnablePipelineArtifactLargeChunkSize = true Source: $(AGENT_ENABLE_PIPELINEARTIFACT_LARGE_CHUNK_SIZE)
Knob: ContinueAfterCancelProcessTreeKillAttempt = true Source: $(VSTSAGENT_CONTINUE_AFTER_CANCEL_PROCESSTREEKILL_ATTEMPT)
Knob: ProcessHandlerTelemetry = true Source: $(AZP_75787_ENABLE_COLLECT)
Knob: ProcessHandlerEnableNewLogic = true Source: $(AZP_75787_ENABLE_NEW_PH_LOGIC)
Knob: IgnoreVSTSTaskLib = true Source: $(AZP_AGENT_IGNORE_VSTSTASKLIB)
Knob: FailJobWhenAgentDies = true Source: $(FAIL_JOB_WHEN_AGENT_DIES)
Knob: CheckForTaskDeprecation = true Source: $(AZP_AGENT_CHECK_FOR_TASK_DEPRECATION)
Knob: MountWorkspace = true Source: $(AZP_AGENT_MOUNT_WORKSPACE)
Finished checking job knob settings.
Start tracking orphan processes.
Finishing: Initialize job
Is it regression?
yes
Expected behavior
I expect this brew installs to work, but after this update, brew installs are failing because these tools install Python and the brew install of python fails to create a bunch of symbolic links mapping previous versions of python to the installed version.
I'm guessing this is related to either the updated version of homebrew, aws-cli, or aws-sam-cli that are in this hosted image update.
I am guessing that something in those package is conflicting with the azure-cli package when it attempts to install python, but it might be related to the PIP/PIPX updates (but I think less likely).
Actual behavior
After this image update 7535fb5
2024-03-07T06:19:43.3360850Z 🍺 /usr/local/Cellar/php/8.3.3_1: 523 files, 86.8MB
2024-03-07T06:19:43.3396350Z ==> Upgrading azure-cli
2024-03-07T06:19:43.3991140Z 2.57.0 -> 2.58.0
2024-03-07T06:19:43.4014650Z ==> Installing dependencies for azure-cli: python@3.11
2024-03-07T06:19:43.4158340Z ==> Installing azure-cli dependency: python@3.11
2024-03-07T06:19:43.4163350Z ==> Downloading https://ghcr.io/v2/homebrew/core/python/3.11/manifests/3.11.8
2024-03-07T06:19:43.4167320Z Already downloaded: /Users/runner/Library/Caches/Homebrew/downloads/eb6ea31a437458e84f35528d6cdf4604a244c9e3ff2f4cbd052bd7e6cabdbd50--python@3.11-3.11.8.bottle_manifest.json
2024-03-07T06:19:43.4372810Z ==> Pouring python@3.11--3.11.8.ventura.bottle.tar.gz
2024-03-07T06:19:51.1988360Z Error: The
brew link
step did not complete successfully2024-03-07T06:19:51.1992480Z The formula built, but is not symlinked into /usr/local
2024-03-07T06:19:51.2000260Z Could not symlink bin/2to3-3.11
2024-03-07T06:19:51.2013540Z Target /usr/local/bin/2to3-3.11
2024-03-07T06:19:51.2018480Z already exists. You may want to remove it:
2024-03-07T06:19:51.2394180Z rm '/usr/local/bin/2to3-3.11'
Repro steps
on the Mac-13 hosted agent, run a bash shell task and a brew install of the following brew packages.
They previously worked, but with the latest devops image update, they are failing to install/link python.
The failure is in installing "azure-cli" which is an indirect dependency that I do not use or have control over installing.
The text was updated successfully, but these errors were encountered: