Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bump go-apk and use faster tarfs implementation #810

Merged
merged 1 commit into from
Nov 1, 2023
Merged

Bump go-apk and use faster tarfs implementation #810

merged 1 commit into from
Nov 1, 2023

Conversation

jonjohnsonjr
Copy link
Contributor

The go-apk bump makes its tarfs implement fs.ReadDirFS so that we can use that directly, which lets us use a cheaper io.ReadAt implementation than copying every file.

@jonjohnsonjr jonjohnsonjr marked this pull request as ready for review October 31, 2023 21:10
@jonjohnsonjr jonjohnsonjr mentioned this pull request Oct 31, 2023
Merged
@jonjohnsonjr jonjohnsonjr marked this pull request as draft October 31, 2023 21:28
auto-merge was automatically disabled October 31, 2023 21:28

Pull request was converted to draft

@jonjohnsonjr
Copy link
Contributor Author

jonjohnsonjr commented Oct 31, 2023 •
edited
Loading

Draft while waiting on chainguard-dev/go-apk#148

time melange lint ~/go-1.21-1.21.0-r0.apk
Linting apk: /Users/jonjohnson/go-1.21-1.21.0-r0.apk
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/github.com/google/pprof/third_party/d3flamegraph/update.sh" as executable: bad magic number '[35 32 32 67]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/golang.org/x/sys/plan9/mkerrors.sh" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/golang.org/x/sys/plan9/mksysnum_plan9.sh" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/golang.org/x/sys/plan9/mkall.sh" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/golang.org/x/sys/unix/mkall.sh" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/golang.org/x/sys/unix/mkerrors.sh" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/golang.org/x/sys/windows/mkknownfolderids.bash" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/golang.org/x/sys/windows/mkerrors.bash" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/compile/profile.sh" as executable: bad magic number '[35 32 67 111]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/compile/internal/ssa/_gen/cover.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/crypto/internal/boring/build.sh" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/crypto/internal/boring/build-goboring.sh" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/crypto/internal/boring/build-boring.sh" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/internal/trace/mkcanned.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/all.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/go/doc/comment/mkstd.sh" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/runtime/race/mkcgo.sh" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/make.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/run.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksysnum_freebsd.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mkerrors.sh" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksysnum_dragonfly.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksysnum_openbsd.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksyscall.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksysnum_netbsd.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksyscall_libc.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksysnum_linux.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksysnum_plan9.sh" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksysctl_openbsd.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mkall.sh" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmp.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/buildall.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/regexp/syntax/make_perl_groups.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/hash/test_gen.awk" as executable: bad magic number '[35 32 67 111]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/clean.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/bootstrap.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/race.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/lib/time/update.bash" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
package linter warning in /Users/jonjohnson/go-1.21-1.21.0-r0.apk: Package contains documentation files but is not a documentation package
melange lint ~/go-1.21-1.21.0-r0.apk  46.97s user 2.14s system 107% cpu 45.834 total

time melange lint ~/go-1.21-1.21.0-r0.apk
Linting apk: /Users/jonjohnson/go-1.21-1.21.0-r0.apk
WARNING: Could not open file "usr/lib/go/lib/time/update.bash" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/all.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/bootstrap.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/buildall.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/clean.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/compile/internal/ssa/_gen/cover.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/compile/profile.sh" as executable: bad magic number '[35 32 67 111]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/github.com/google/pprof/third_party/d3flamegraph/update.sh" as executable: bad magic number '[35 32 32 67]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/golang.org/x/sys/plan9/mkall.sh" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/golang.org/x/sys/plan9/mkerrors.sh" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/golang.org/x/sys/plan9/mksysnum_plan9.sh" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/golang.org/x/sys/unix/mkall.sh" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/golang.org/x/sys/unix/mkerrors.sh" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/golang.org/x/sys/windows/mkerrors.bash" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmd/vendor/golang.org/x/sys/windows/mkknownfolderids.bash" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/cmp.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/crypto/internal/boring/build-boring.sh" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/crypto/internal/boring/build-goboring.sh" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/crypto/internal/boring/build.sh" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/go/doc/comment/mkstd.sh" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/hash/test_gen.awk" as executable: bad magic number '[35 32 67 111]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/internal/trace/mkcanned.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/make.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/race.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/regexp/syntax/make_perl_groups.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/run.bash" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/runtime/race/mkcgo.sh" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mkall.sh" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mkerrors.sh" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksyscall.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksyscall_libc.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksysctl_openbsd.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksysnum_dragonfly.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksysnum_freebsd.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksysnum_linux.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksysnum_netbsd.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksysnum_openbsd.pl" as executable: bad magic number '[35 33 47 117]' in record at byte 0x0
WARNING: Could not open file "usr/lib/go/src/syscall/mksysnum_plan9.sh" as executable: bad magic number '[35 33 47 98]' in record at byte 0x0
package linter warning in /Users/jonjohnson/go-1.21-1.21.0-r0.apk: Package contains documentation files but is not a documentation package
melange lint ~/go-1.21-1.21.0-r0.apk  1.12s user 0.25s system 78% cpu 1.736 total

@jonjohnsonjr jonjohnsonjr marked this pull request as ready for review October 31, 2023 21:32
@jonjohnsonjr jonjohnsonjr marked this pull request as draft October 31, 2023 22:01
@jonjohnsonjr
Bump go-apk and use faster tarfs implementation
9aa0800 
The go-apk bump makes its tarfs implement fs.ReadDirFS so that we can
use that directly, which lets us use a cheaper io.ReadAt implementation
than copying every file.

Signed-off-by: Jon Johnson <jon.johnson@chainguard.dev>
@jonjohnsonjr jonjohnsonjr marked this pull request as ready for review November 1, 2023 15:28
@jonjohnsonjr jonjohnsonjr merged commit f76397c into chainguard-dev:main Nov 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@imjasonh imjasonh imjasonh approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jonjohnsonjr @imjasonh