Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GitError Class:Zlib on cloning General #7

Closed
c42f opened this issue Oct 19, 2020 · 9 comments · Fixed by #13
Closed

GitError Class:Zlib on cloning General #7

c42f opened this issue Oct 19, 2020 · 9 comments · Fixed by #13
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@c42f
Copy link

c42f commented Oct 19, 2020

I've been fairly frequently getting a mysterious GitError(Code:ERROR, Class:Zlib, error inflating zlib stream) on julia-nightly while using julia-buildpkg.

I think this might possibly only occur on Juila-nightly. Example:

https://github.com/JuliaArrays/StaticArrays.jl/pull/840/checks?check_run_id=1272952752

Run julia-actions/julia-buildpkg@latest
     Cloning registry from "https://github.com/JuliaRegistries/General.git"
25l    Fetching: [>                                        ]  0.0 %
    Fetching: [==========>                              ]  22.5 %
    Fetching: [================>                        ]  38.2 %
    Fetching: [=========================>               ]  60.5 %
ERROR: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:Zlib, error inflating zlib stream)
Stacktrace:
  [1] pkgerror(msg::String)
    @ Pkg.Types /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:52
  [2] clone(ctx::Pkg.Types.Context, url::String, source_path::String; header::String, credentials::Nothing, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.GitTools /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/GitTools.jl:153
  [3] (::Pkg.Types.var"#82#85"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec})(tmp::String)
    @ Pkg.Types /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:998
  [4] mktempdir(fn::Pkg.Types.var"#82#85"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec}, parent::String; prefix::String)
    @ Base.Filesystem ./file.jl:711
  [5] mktempdir
    @ ./file.jl:709 [inlined]
  [6] clone_or_cp_registries(ctx::Pkg.Types.Context, regs::Vector{Pkg.Types.RegistrySpec}, depot::String)
    @ Pkg.Types /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:978
  [7] clone_or_cp_registries
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:971 [inlined]
  [8] #add#6
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:32 [inlined]
  [9] add
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:28 [inlined]
 [10] #add#5
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:26 [inlined]
 [11] add
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:26 [inlined]
 [12] add(regs::Vector{String}; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.Registry /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:25
 [13] add
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:25 [inlined]
 [14] #add#1
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:24 [inlined]
 [15] add(reg::String)
    @ Pkg.Registry /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:24
 [16] top-level scope
    @ none:1

caused by: GitError(Code:ERROR, Class:Zlib, error inflating zlib stream)
Stacktrace:
  [1] macro expansion
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/LibGit2/src/error.jl:110 [inlined]
  [2] clone(repo_url::SubString{String}, repo_path::String, clone_opts::LibGit2.CloneOptions)
    @ LibGit2 /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/LibGit2/src/repository.jl:459
  [3] clone(repo_url::SubString{String}, repo_path::String; branch::String, isbare::Bool, remote_cb::Ptr{Nothing}, credentials::LibGit2.CachedCredentials, callbacks::Dict{Symbol, Tuple{Ptr{Nothing}, Any}})
    @ LibGit2 /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/LibGit2/src/LibGit2.jl:580
  [4] clone(ctx::Pkg.Types.Context, url::String, source_path::String; header::String, credentials::Nothing, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.GitTools /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/GitTools.jl:143
  [5] (::Pkg.Types.var"#82#85"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec})(tmp::String)
    @ Pkg.Types /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:998
  [6] mktempdir(fn::Pkg.Types.var"#82#85"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec}, parent::String; prefix::String)
    @ Base.Filesystem ./file.jl:711
  [7] mktempdir
    @ ./file.jl:709 [inlined]
  [8] clone_or_cp_registries(ctx::Pkg.Types.Context, regs::Vector{Pkg.Types.RegistrySpec}, depot::String)
    @ Pkg.Types /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:978
  [9] clone_or_cp_registries
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:971 [inlined]
 [10] #add#6
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:32 [inlined]
 [11] add
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:28 [inlined]
 [12] #add#5
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:26 [inlined]
 [13] add
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:26 [inlined]
 [14] add(regs::Vector{String}; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.Registry /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:25
 [15] add
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:25 [inlined]
 [16] #add#1
    @ /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:24 [inlined]
 [17] add(reg::String)
    @ Pkg.Registry /buildworker/worker/package_linux32/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:24
 [18] top-level scope
    @ none:1
25h
Error: Process completed with exit code 1.
@c42f
Copy link
Author

c42f commented Oct 19, 2020

May possibly be related to the fact that we're using libgit2 for this

#6

And julia nightly was upgraded to use libgit2-1.0.0

JuliaLang/julia#35233

which has a bug which also bit the rust people and seems like it could be similar

rust-lang/cargo#8258

I tried to reproduce outside of julia-buildpkg, but haven't had any luck so far. Here's the script I'm using:

#!/bin/bash

export JULIA_PKG_SERVER=""
export JULIA_DEPOT_PATH=$(mktemp -d)
trap "rm -Rf $JULIA_DEPOT_PATH" EXIT

julia --color=yes -e 'using Pkg; Pkg.Registry.add("General")'

@c42f
Copy link
Author

c42f commented Oct 19, 2020 •
edited
Loading

By the way, I've seen this bug in nightly CI for more than one package.

@c42f c42f mentioned this issue Oct 19, 2020
Closed
@SaschaMann SaschaMann added help wanted Extra attention is needed bug Something isn't working labels Oct 28, 2020
@SaschaMann
Copy link
Member

SaschaMann commented Oct 28, 2020 •
edited
Loading

Could you rerun the build and see if it still happens?

I asked on Slack and Elliot said it may have been fixed in libgit2 1.0.1, which Julia nightly should be using now.

@c42f
Copy link
Author

c42f commented Oct 30, 2020

Rerunning it appears this may have been fixed. I'll reopen if I see it again.

@c42f c42f closed this as completed Oct 30, 2020
@c42f
Copy link
Author

c42f commented Nov 4, 2020

I've just seen this error again:

https://github.com/JuliaArrays/StaticArrays.jl/pull/848/checks?check_run_id=1351636719

@c42f c42f reopened this Nov 4, 2020
@c42f c42f mentioned this issue Nov 4, 2020
Merged
@SaschaMann
Copy link
Member

I have no idea what causes this, sorry. Perhaps open an issue on Pkg and see if they have an idea?

@DilumAluthge
Copy link
Member

It seems that when this happens, it is intermittent. So I think something like #13 should fix this.

@DilumAluthge DilumAluthge mentioned this issue Jan 2, 2021
Merged
@DilumAluthge
Copy link
Member

Also, even if cloning General fails, we should still continue the build. This is what #13 does.

@c42f
Copy link
Author

c42f commented Jan 4, 2021

Thanks #13 looks good. I haven't had time to look into this further or figure out how to reproduce.

@chenwilliam77 chenwilliam77 mentioned this issue Jan 30, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Put the General registry clone inside a bounded retry loop julia-actions/julia-buildpkg
3 participants
@c42f @DilumAluthge @SaschaMann