Reapply "[image-builder-bob] bump up buildkit (#20690)" (#20693) #20694
Conversation
iQQBot
force-pushed
the
pd/upgrade-buildkit
branch
from
d5d1291 to
b568af2
Compare
| if strings.HasPrefix(dest, "/proc/thread-self/") { | ||
| target = filepath.Join("/proc", strconv.Itoa(int(req.Pid)), strings.TrimPrefix(dest, "/proc/thread-self/")) | ||
| } |
There was a problem hiding this comment.
How does this change impact regular workspaces, and prebuilds?
There was a problem hiding this comment.
I see tests were failing, and assume this is the intended fix:
--- FAIL: TestBaseImageBuild (50.03s)
--- FAIL: TestBaseImageBuild/database (0.00s)
--- FAIL: TestBaseImageBuild/database/it_should_build_a_base_image (50.03s)
--- FAIL: TestParallelBaseImageBuild (28.78s)
--- FAIL: TestParallelBaseImageBuild/image-builder (0.00s)
--- FAIL: TestParallelBaseImageBuild/image-builder/it_should_allow_parallel_build_of_images (28.78s)
There was a problem hiding this comment.
It would be good, I think, to get a review for this file from @Furisto @aledbf or @csweichel .
There was a problem hiding this comment.
We previously did not handle thread-self, so it would fallback to nsenter, using the tid/pid of nsenter at that time. Handling it here actually increases some security.
There was a problem hiding this comment.
@iQQBot we were talking earlier today, my recollection is, something in buildkit must have changed and is now relying on threading, whereas previously it was not. However, we don't know what that is, is that right?
May I ask, how did you find this change was necessary? As in, what led you to make this change? Can you link to something that tipped you off / pointed you in the right direction? I ask because I'm curious how you did the related debugging. For example, I am not sure if you saw something with journalct or dmesg and wish to know more.
I found upload_parallelism was added:
moby/buildkit@22f6b3e#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R581
It mentions:
Each individual layer is uploaded with 5 threads, using the Upload manager provided by the AWS SDK.
Perhaps that is it?
I see we use many gorountimes and then upload here.
There was a problem hiding this comment.
Image build fail if not change this
|
/unhold |
This reverts commit 7137833.
Tool: gitpod/catfood.gitpod.cloud
Description
This pid is actually thread id, it should be safe if we use in
thread-selfsee doc link
workspace integration test are success https://github.com/gitpod-io/gitpod/actions/runs/13993623337/job/39183946713?pr=20694
Related Issue(s)
Fixes CLC-1252
How to test
Documentation
Preview status
Gitpod was successfully deployed to your preview environment.
Build Options
Build
Run the build with werft instead of GHA
Run Leeway with
--dont-testPublish
Installer
Add desired feature flags to the end of the line above, space separated
Preview Environment / Integration Tests
If enabled this will build
install/previewIf enabled this will create the environment on GCE infra
Saves cost. Untick this only if you're really sure you need a non-preemtible machine.
Valid options are
all,workspace,webapp,ide,jetbrains,vscode,ssh. If enabled,with-previewandwith-large-vmwill be enabled./hold