Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[New curl version] 8.7.1 #4883

Closed
github-actions bot opened this issue Mar 27, 2024 · 10 comments · Fixed by git-for-windows/MSYS2-packages#165 or git-for-windows/MINGW-packages#117
Closed

[New curl version] 8.7.1 #4883

github-actions bot opened this issue Mar 27, 2024 · 10 comments · Fixed by git-for-windows/MSYS2-packages#165 or git-for-windows/MINGW-packages#117

Comments

@github-actions
Copy link

https://github.com/curl/curl/releases/tag/curl-8_7_1

@dscho
Copy link
Member

dscho commented Mar 27, 2024

Let's open the PRs just in case, but wait with deploying them for a few seconds 😁

@dscho
Copy link
Member

dscho commented Mar 27, 2024

/open pr

The MSYS workflow run was started

The MINGW workflow run was started

@dscho
Copy link
Member

dscho commented Mar 27, 2024

Image

Not yet:

 curl: (22) The requested URL returned error: 404
==> ERROR: Failure while downloading https://github.com/curl/curl/releases/download/curl-8_7_1/curl-8.7.1.tar.bz2

@rimrul
Copy link
Member

rimrul commented Mar 27, 2024

/open pr

The MSYS workflow run was started

The MINGW workflow run was started

@rimrul
Copy link
Member

rimrul commented Mar 30, 2024

Let's open the PRs just in case, but wait with deploying them for a few seconds 😁

We should probably wait and see what comes of https://lore.kernel.org/git/20240330000212.GA1261238@coredump.intra.peff.net/T/#u (curl/curl#13229)

@rimrul
Copy link
Member

rimrul commented Apr 9, 2024

I'm not quite sure how best to handle this. Possible options include

@dscho
Copy link
Member

dscho commented Apr 9, 2024

My preference would be to do both of these:

FWIW my hesitance to go forward with deploying cURL is solely due to those hangs on the Windows/ARM64 runners. I still have no idea how to fix the MSYS2 runtime (and I lack the time to reduce the rather unwieldy pacman reproducer into a neat and small reproducer).

My current thinking is that we could add a work-around for the time being:

  • add a step in the build-and-deploy workflow that
    • only runs on Windows/ARM64
    • spawns a PowerShell script that runs detached while the workflow continues with the next steps of the job,
    • this PowerShell script would basically loop, and in the loop body it would
      • see if there is any new pacman.exe process that has the exact same command-line as its parent,
      • see such a process has been "alive" for more than, say, 30 seconds, and if so, stop it,
      • sleep something like 10 seconds before continuing with the next loop iteration

The PowerShell script would be a variation of what I used when I logged interactively to unstick builds in the past:

$p = (Get-CimInstance Win32_Process |
  Where-Object {$_.CommandLine -like '*pacman*'} |
  Select-Object CommandLine, ParentProcessId, ProcessId |
  Group-Object CommandLine |
  Where-Object {$_.Count -gt 1} |
  Select-Object -ExpandProperty Group)
if ($p.Length -eq 2 -and $p[0].processId -eq $p[1].parentProcessId) {
  Stop-Process -Force -Id $p[1].processId
}

There's still work to do here: Instead of stopping the process immediately, we should check that:

  • It has been running for a while (30 seconds should be plenty, if this is indeed just some shim process that is waiting for a gpg process that is long gone).
  • Ideally, we would find a way to determine whether the process itself is idle. One option would be to look at (Get-Process -Id $pid).CPU (which stores the total amount of CPU seconds spent on that process) and require it to change only a fraction over the course of half a minute or so.

@dscho
Copy link
Member

dscho commented Apr 9, 2024

My preference would be to do both of these:

This is prepared via git-for-windows/MINGW-packages@e5725f2

And this via #4906.

dscho added a commit that referenced this issue Apr 10, 2024
As discussed
[here](#4883 (comment)),
we should have a work-around in place before deploying libcurl 8.7.1.
git-for-windows-ci pushed a commit that referenced this issue Apr 10, 2024
As discussed
[here](#4883 (comment)),
we should have a work-around in place before deploying libcurl 8.7.1.
git-for-windows-ci pushed a commit that referenced this issue Apr 10, 2024
As discussed
[here](#4883 (comment)),
we should have a work-around in place before deploying libcurl 8.7.1.
git-for-windows-ci pushed a commit that referenced this issue Apr 12, 2024
As discussed
[here](#4883 (comment)),
we should have a work-around in place before deploying libcurl 8.7.1.
git-for-windows-ci pushed a commit that referenced this issue Apr 12, 2024
As discussed
[here](#4883 (comment)),
we should have a work-around in place before deploying libcurl 8.7.1.
@dscho dscho added this to the Next release milestone Apr 12, 2024
git-for-windows-ci pushed a commit that referenced this issue Apr 12, 2024
As discussed
[here](#4883 (comment)),
we should have a work-around in place before deploying libcurl 8.7.1.
dscho added a commit that referenced this issue Apr 14, 2024
As discussed
[here](#4883 (comment)),
we should have a work-around in place before deploying libcurl 8.7.1.
dscho added a commit to git-for-windows/git-sdk-arm64 that referenced this issue May 3, 2024
For some reason, there are relatively consistent hangs on Windows/ARM64
when trying to run `pacman.exe` multiple times. Terminating the "right"
processes seems to unblock those hangs (without failure!)

The most likely explanation is a dead-lock in the (x86_64) MSYS2 runtime
where the process that handles signals for child processes is waiting
for that child process, but it is long gone, and the signal handler
process waits forever, blocking its parent process forever.

To help with that, let's implements the PowerShell script outlined in
git-for-windows/git#4883 (comment)
that identifies those hanging processes, waits for them to be idle just
to be extra certain not to disrupt any regular Pacman operation that
waits for a good reason.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit to git-for-windows/git-sdk-arm64 that referenced this issue May 3, 2024
For some reason, there are relatively consistent hangs on Windows/ARM64
when trying to run `pacman.exe` multiple times. Terminating the "right"
processes seems to unblock those hangs (without failure!)

The most likely explanation is a dead-lock in the (x86_64) MSYS2 runtime
where the process that handles signals for child processes is waiting
for that child process, but it is long gone, and the signal handler
process waits forever, blocking its parent process forever.

To help with that, let's implements the PowerShell script outlined in
git-for-windows/git#4883 (comment)
that identifies those hanging processes, waits for them to be idle just
to be extra certain not to disrupt any regular Pacman operation that
waits for a good reason.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit to git-for-windows/git-sdk-arm64 that referenced this issue May 3, 2024
For some reason, there are relatively consistent hangs on Windows/ARM64
when trying to run `pacman.exe` multiple times. Terminating the "right"
processes seems to unblock those hangs (without failure!)

The most likely explanation is a dead-lock in the (x86_64) MSYS2 runtime
where the process that handles signals for child processes is waiting
for that child process, but it is long gone, and the signal handler
process waits forever, blocking its parent process forever.

To help with that, let's implements the PowerShell script outlined in
git-for-windows/git#4883 (comment)
that identifies those hanging processes, waits for them to be idle just
to be extra certain not to disrupt any regular Pacman operation that
waits for a good reason.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@dscho
Copy link
Member

dscho commented May 3, 2024

My current thinking is that we could add a work-around for the time being:

* add a step in the `build-and-deploy` workflow that
  
  * only runs on Windows/ARM64
  * spawns a PowerShell script that runs detached while the workflow continues with the next steps of the job,
  * this PowerShell script would basically loop, and in the loop body it would
    
    * see if there is any new `pacman.exe` process that has the exact same command-line as its parent,
    * see such a process has been "alive" for more than, say, 30 seconds, and if so, stop it,
    * sleep something like 10 seconds before continuing with the next loop iteration

Hmm. This seems not to have worked as well as I hoped.

@dscho
Copy link
Member

dscho commented May 5, 2024

This will hopefully provide the work-around we need to proceed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants