Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set build type for scheduled builds based on doc repo changes #2931

Merged
merged 36 commits into from
Feb 9, 2024
Merged

Set build type for scheduled builds based on doc repo changes #2931

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
0bb4951
check if docs repo has changed
bmorelli25 Feb 8, 2024
eaa67df
more
bmorelli25 Feb 8, 2024
0ea95ca
updates so I can test it
bmorelli25 Feb 9, 2024
9694e32
build commit
bmorelli25 Feb 9, 2024
4ef9614
lbc
bmorelli25 Feb 9, 2024
92df131
gpb
bmorelli25 Feb 9, 2024
dfcd5ab
more testing
bmorelli25 Feb 9, 2024
3b563f5
a
bmorelli25 Feb 9, 2024
a4649fc
b
bmorelli25 Feb 9, 2024
b50be86
mmhm
bmorelli25 Feb 9, 2024
132847e
use auth token
bmorelli25 Feb 9, 2024
b06de72
fix quotes
bmorelli25 Feb 9, 2024
b715781
c
bmorelli25 Feb 9, 2024
fea9ced
aha!
bmorelli25 Feb 9, 2024
215e20d
no way this works
bmorelli25 Feb 9, 2024
b1bf2ae
forgot the squigles
bmorelli25 Feb 9, 2024
840bd3c
back up
bmorelli25 Feb 9, 2024
0f64c00
perhaps
bmorelli25 Feb 9, 2024
65c95cc
lsc
bmorelli25 Feb 9, 2024
9e266fe
no squigs
bmorelli25 Feb 9, 2024
57deb0c
jq isn’t working
bmorelli25 Feb 9, 2024
25ab6a6
quotes
bmorelli25 Feb 9, 2024
fcf36d5
try a file
bmorelli25 Feb 9, 2024
9c533a5
ugh
bmorelli25 Feb 9, 2024
14ff55a
d
bmorelli25 Feb 9, 2024
0abdfa7
e
bmorelli25 Feb 9, 2024
163784a
move to file, echo all the things
bmorelli25 Feb 9, 2024
0e55959
Fixing up the script - mostly adding curl and jq flags
nkammah Feb 9, 2024
a285d0d
wire up the rebuild
nkammah Feb 9, 2024
bd23c64
fix script typo
nkammah Feb 9, 2024
94070db
export rebuild var
nkammah Feb 9, 2024
5bc5c78
Make the script more generic
nkammah Feb 9, 2024
164ef96
Set REBUILD value to false since empty strings are not accepted
nkammah Feb 9, 2024
1b9b2af
Remove early exits used for testing purposes
nkammah Feb 9, 2024
c3e8250
Try setting concurrency group name based on branch
nkammah Feb 9, 2024
5b3cadb
uncomment
bmorelli25 Feb 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .buildkite/build_pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ steps:
value: ""
hint: "Should we ignore checking broken links? Should we allow to run the build without failing if there's a broken link? Ignoring broken links is dangerous not just because bad links will leak into the public site but because subsequent builds and pull requests that do not fix the links fail."
- wait
- label: "Full rebuild or incremental build?"
if: build.source == "schedule"
command: ".buildkite/scripts/compare_commits.sh"
Copy link
Member Author

@bmorelli25 bmorelli25 Feb 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nkammah why does L30 not need bash in the command but L35 does?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We specify the script interpreter in the first line of the script here - that's what we call the "shebang". If the script is executable (ex. running "chmod a+x" on it), the pipeline will know how to call it.

As for the discrepancy on L35, it could be legacy, or that we explicitly specified it at the time. Both approaches will work.

- label: ":white_check_mark: Build docs"
command: |
export REBUILD="$(buildkite-agent meta-data get REBUILD --default '' --log-level fatal)"
Expand All @@ -34,7 +37,7 @@ steps:
provider: "gcp"
image: family/docs-ubuntu-2204
machineType: ${BUILD_MACHINE_TYPE}
concurrency_group: build-docs
concurrency_group: build-docs-${BUILDKITE_BRANCH}
bmorelli25 marked this conversation as resolved.
Show resolved Hide resolved
concurrency: 1
notify:
- email: "docs-status@elastic.co"
Expand Down
1 change: 1 addition & 0 deletions .buildkite/hooks/pre-command
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ if [[ "$BUILDKITE_PIPELINE_SLUG" == "docs-build-pr" ]];then
fi
elif [[ "$BUILDKITE_PIPELINE_SLUG" == "docs-build" ]];then
export BUILD_MACHINE_TYPE="n2-highcpu-32"
export BUILDKITE_API_TOKEN=$(retry 5 vault kv get -field=value secret/ci/elastic-docs/buildkite_token)
elif [[ "$BUILDKITE_PIPELINE_SLUG" == "docs-build-air-gapped" ]] && [[ "$BUILDKITE_STEP_KEY" == "publish-air-gapped-doc" ]]; then
export DOCKER_USERNAME=$(retry 5 vault kv get -field=username secret/ci/elastic-docs/docker.elastic.co)
export DOCKER_PASSWORD=$(retry 5 vault kv get -field=password secret/ci/elastic-docs/docker.elastic.co)
Expand Down
13 changes: 13 additions & 0 deletions .buildkite/scripts/compare_commits.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/bin/bash

last_successful_build_url="https://api.buildkite.com/v2/organizations/elastic/pipelines/${BUILDKITE_PIPELINE_SLUG}/builds?branch=${BUILDKITE_BRANCH}&state=passed"
LAST_SUCCESSFUL_COMMIT=$(curl -s -H "Authorization: Bearer ${BUILDKITE_API_TOKEN}" $last_successful_build_url | jq -r '.[0].commit')

echo "Comparing the current docs build commit ${BUILDKITE_COMMIT} to the last successful build commit ${LAST_SUCCESSFUL_COMMIT}"
if [[ "$BUILDKITE_COMMIT" == "$LAST_SUCCESSFUL_COMMIT" ]]; then
echo "The docs repo has not changed since the last build."
buildkite-agent meta-data set "REBUILD" "false"
else
echo "The docs repo has changed since the last build."
buildkite-agent meta-data set "REBUILD" "rebuild"
fi