Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

features: installer artifacts build paralleled, do not build up-to-date images #39

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open

features: installer artifacts build paralleled, do not build up-to-date images #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
c11dcb2
it seems ADDITIONAL_BIN_FOLDER_FROM_REPO is used instedof ADDITIONAL_…
Apr 21, 2023
4744734
Merge pull request #1 from openshift-mirror/ds01
Apr 21, 2023
325f5d7
it seems ADDITIONAL_BIN_FOLDER_FROM_REPO is used instedof ADDITIONAL_…
Apr 21, 2023
780a2dc
it seems ADDITIONAL_BIN_FOLDER_FROM_REPO is used instedof ADDITIONAL_…
Apr 21, 2023
03ec707
initial commit of installer artifacts build sequence split
Jun 5, 2023
0572ad6
Merge branch 'artifact-build-split' into split-merge
Jun 5, 2023
8e5d3d3
4.14 -> 4.12
Jun 5, 2023
a121892
go 1.20 -> 1.19
Jun 5, 2023
893de12
TARGET -> TGT
Jun 5, 2023
887234e
debug
Jun 5, 2023
f1d2604
debug2
Jun 5, 2023
055af53
debug3
Jun 5, 2023
29e322d
fix:
Jun 5, 2023
85db2bb
temporary image stream for artifacts
Jun 5, 2023
9dd07ac
bc names become more accurate
Jun 6, 2023
a457014
bc names become more accurate again
Jun 6, 2023
550aeee
add: OCP_VERSION to buildArgs to avoid defaulting from dockerfile
Jun 6, 2023
7c42356
bc reodred in pipeline tasks due to splitting installer-artifacts bui…
Jun 6, 2023
9ad962d
Merge branch 'split-merge' into gitbucket-main-split-merge
Jun 6, 2023
f545ee2
fix: wrong repo
Jun 7, 2023
3dba42c
build outdated images feature added
Jun 7, 2023
0b1f1c2
fix: wrong task index
Jun 9, 2023
5d9c23e
fix: wrong identation
Jun 9, 2023
1fa4dcb
avoid butter oil
Jun 9, 2023
ec0f41b
merge upstream
Jun 11, 2023
61b37c1
okd-rpms reworked to be a hard links for neseccary rpms at release:ar…
Jun 11, 2023
68344c0
always rebuild the resulting image since the included might have changed
Jun 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions buildconfigs/01-imagestream.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ spec:
lookupPolicy:
local: true
tags:
- name: okd-rpms
from:
kind: ImageStreamTag
name: release:artifacts
- name: kuryr-cni
from:
kind: DockerImage
Expand Down
24 changes: 16 additions & 8 deletions buildconfigs/02-additional-imagestream.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
apiVersion: image.openshift.io/v1
kind: ImageStream
metadata:
Expand All @@ -14,11 +15,18 @@ spec:
type: Local
importPolicy:
importMode: PreserveOriginal # Needs OKD 4.13
- name: fedora36
from:
kind: DockerImage
name: 'quay.io/fedora/fedora:36'
referencePolicy:
type: Local
importPolicy:
importMode: PreserveOriginal # Needs OKD 4.13
# upstream already resides on fedora:37 (Dockerfile.okd -> Dockerfile.fcos)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add the fedora:3[78] ISTags.

# while scos is being built by Dockerfile.scos
# - name: fedora36
# from:
# kind: DockerImage
# name: 'quay.io/fedora/fedora:36'
# referencePolicy:
# type: Local
# importPolicy:
# importMode: PreserveOriginal # Needs OKD 4.13
---
apiVersion: image.openshift.io/v1
kind: ImageStream
metadata:
name: artifacts
1 change: 1 addition & 0 deletions buildconfigs/03-forked-dockerfiles.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ kind: BuildConfig
metadata:
name: forked-dockerfiles
labels:
builder-replacement: skip
skip-release-branch: "true"
skip-replace-cli: "true"
spec:
Expand Down
1 change: 1 addition & 0 deletions buildconfigs/05-base.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ kind: BuildConfig
metadata:
name: base
labels:
builder-replacement: skip
# TODO: do we need branch cut for this?
skip-release-branch: "true"
skip-replace-cli: "true"
Expand Down
4 changes: 2 additions & 2 deletions buildconfigs/06-cli.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,7 @@ spec:
- name: ARCHITECTURES
value: ""
- name: MAKE_ARGS
value: >-
-e --warn-undefined-variables
value: -e --warn-undefined-variables
- name: INSTALL_PKGS
value: "krb5-devel"
- name: INSTALL_PKGS_RUNTIME
Expand All @@ -57,6 +56,7 @@ spec:
value: >
OpenShift is a platform for developing,
building, and deploying containerized applications.
# are these tag really necessary here?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I remember correctly these are replaced later with the okd ones and are needed for some specific code to be considered for OKD vs OCP vs OKD/SCOS. I'm not sure if this is needed for the cli too though, @vrutkovs ?

- name: TAGS
value: openshift,cli
- name: VERSIONS
Expand Down
1 change: 1 addition & 0 deletions buildconfigs/10-baremetal-installer.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ spec:
imageOptimizationPolicy: SkipLayers
dockerfilePath: images/baremetal/Dockerfile.centos9
buildArgs:
# are these tag really necessary here?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

- name: "TAGS"
value: "baremetal libvirt okd"
from:
Expand Down
4 changes: 2 additions & 2 deletions buildconfigs/10-cli-artifacts.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,7 @@ spec:
- name: ARCHITECTURES
value: "x86_64 aarch64 s390x ppc64le"
- name: MAKE_ARGS
value: >-
cross-build --warn-undefined-variables
value: "cross-build --warn-undefined-variables"
- name: INSTALL_PKGS
value: "krb5-devel"
- name: INSTALL_PKGS_RUNTIME
Expand All @@ -52,6 +51,7 @@ spec:
value: >
OpenShift is a platform for developing,
building, and deploying containerized applications.
# are these tag really necessary here?
- name: TAGS
value: openshift,cli
- name: VERSIONS
Expand Down
1 change: 1 addition & 0 deletions buildconfigs/10-hyperkube.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ spec:
value: >
OpenShift is a platform for developing,
building, and deploying containerized applications.
# are these tag really necessary here?
- name: TAGS
value: openshift,hyperkube
- name: VERSIONS
Expand Down
24 changes: 24 additions & 0 deletions buildconfigs/10-installer-artifact-amd64-lnx.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
name: installer-artifact-amd64-lnx
labels:
part-of-artifacts: installer
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is ideal at this time. Deviating from the prow builds definitions may be a future step. However, the maintenance of the repo might become harder including these changes..

spec:
source:
type: Git
git:
uri: 'https://github.com/openshift/installer'
ref: release-4.12
images: []
strategy:
type: Docker
dockerStrategy:
buildArgs:
- name: "TARGET_ARCH"
value: "amd64"
- name: "TARGET_OS"
value: "linux"
output:
to:
name: 'artifacts:installer-amd64-lnx'
24 changes: 24 additions & 0 deletions buildconfigs/10-installer-artifact-amd64-mac.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
name: installer-artifact-amd64-mac
labels:
part-of-artifacts: installer
spec:
source:
type: Git
git:
uri: 'https://github.com/openshift/installer'
ref: release-4.12
images: []
strategy:
type: Docker
dockerStrategy:
buildArgs:
- name: "TARGET_ARCH"
value: "amd64"
- name: "TARGET_OS"
value: "darwin"
output:
to:
name: 'artifacts:installer-amd64-mac'
24 changes: 24 additions & 0 deletions buildconfigs/10-installer-artifact-arm64-lnx.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
name: installer-artifact-arm64-lnx
labels:
part-of-artifacts: installer
spec:
source:
type: Git
git:
uri: 'https://github.com/openshift/installer'
ref: release-4.12
images: []
strategy:
type: Docker
dockerStrategy:
buildArgs:
- name: "TARGET_ARCH"
value: "arm64"
- name: "TARGET_OS"
value: "linux"
output:
to:
name: 'artifacts:installer-arm64-lnx'
24 changes: 24 additions & 0 deletions buildconfigs/10-installer-artifact-arm64-mac.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
name: installer-artifact-arm64-mac
labels:
part-of-artifacts: installer
spec:
source:
type: Git
git:
uri: 'https://github.com/openshift/installer'
ref: release-4.12
images: []
strategy:
type: Docker
dockerStrategy:
buildArgs:
- name: "TARGET_ARCH"
value: "arm64"
- name: "TARGET_OS"
value: "darwin"
output:
to:
name: 'artifacts:installer-arm64-mac'
30 changes: 0 additions & 30 deletions buildconfigs/10-installer-artifacts.yaml
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions buildconfigs/10-installer.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ spec:
imageOptimizationPolicy: SkipLayers
dockerfilePath: images/installer/Dockerfile.ci
buildArgs:
# are those tag really necessary here? seems "no"
- name: "TAGS"
value: "okd"
from:
Expand Down
10 changes: 5 additions & 5 deletions buildconfigs/10-ironic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@ spec:
ref: release-4.12
contextDir: .
images:
- from:
kind: ImageStreamTag
name: 'tools:fedora36'
as:
- 'fedora:35'
# - from:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either delete or update, I'd avoid commenting. I'd prefer updating as per the previous comment.

# kind: ImageStreamTag
# name: 'tools:fedora36'
# as:
# - 'fedora:35'
- from:
kind: ImageStreamTag
name: 'tools:centos9'
Expand Down
1 change: 1 addition & 0 deletions buildconfigs/10-machine-config-operator.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ spec:
imageOptimizationPolicy: SkipLayers
dockerfilePath: Dockerfile
buildArgs:
# are these tag really necessary here? seems "no"
- name: "TAGS"
value: "okd"
from:
Expand Down
2 changes: 1 addition & 1 deletion buildconfigs/10-network-interface-bond-cni.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
name: bond-cni
name: network-interface-bond-cni
spec:
source:
type: Git
Expand Down
1 change: 1 addition & 0 deletions buildconfigs/10-ovirt-installer.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ spec:
imageOptimizationPolicy: SkipLayers
dockerfilePath: images/installer/Dockerfile.ci
buildArgs:
# are those tag really necessary here? seems no
- name: "TAGS"
value: "okd"
from:
Expand Down
41 changes: 36 additions & 5 deletions buildconfigs/20-artifacts.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,17 @@ kind: BuildConfig
metadata:
name: artifacts
labels:
builder-replacement: skip
change-build-arg-tags: "true"
meta-for: installer
resources-profile: low
skip-release-branch: "true"
skip-replace-cli: "true"
spec:
output:
to:
kind: ImageStreamTag
name: 'release:artifacts'
source:
contextDir: .
images:
Expand All @@ -18,6 +27,31 @@ spec:
name: 'release:hyperkube'
as:
- 'release:hyperkube'
- as:
- artifacts:installer-amd64-lnx
from:
kind: ImageStreamTag
name: artifacts:installer-amd64-lnx
- as:
- artifacts:installer-amd64-mac
from:
kind: ImageStreamTag
name: artifacts:installer-amd64-mac
- as:
- artifacts:installer-arm64-lnx
from:
kind: ImageStreamTag
name: artifacts:installer-arm64-lnx
- as:
- artifacts:installer-arm64-mac
from:
kind: ImageStreamTag
name: artifacts:installer-arm64-mac
#- from:
# kind: ImageStreamTag
# name: 'release:installer-artifacts'
# as:
# - 'release:installer-artifacts'
- from:
kind: ImageStreamTag
name: 'release:forked-dockerfiles'
Expand All @@ -27,12 +61,9 @@ spec:
strategy:
type: Docker
dockerStrategy:
imageOptimizationPolicy: SkipLayers
buildArgs: []
dockerfilePath: Dockerfile.centos9
from:
kind: "ImageStreamTag"
name: "release:base"
output:
to:
kind: ImageStreamTag
name: 'release:artifacts'
imageOptimizationPolicy: SkipLayers
Loading