Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PMM-7 Remove go tip runs, use go version for CI from go.mod file #2137

Merged
merged 14 commits into from
May 18, 2023
Merged

PMM-7 Remove go tip runs, use go version for CI from go.mod file #2137

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
6ff8e87
PMM-7 Remove go tip runs, use go version for CI from go.mod file
artemgavrilov May 16, 2023
0cf7705
PMM-7 Disable embedded cache
artemgavrilov May 16, 2023
1bd87e1
PMM-7 Cleanup go tip builds, reconfigure setup-go
artemgavrilov May 16, 2023
f00b8e9
PMM-7 Fix
artemgavrilov May 16, 2023
1c44e8f
Merge remote-tracking branch 'origin/main' into PMM-7-fix-ci-caches
artemgavrilov May 16, 2023
0196834
PMM-7 Use verbose mode for go mod download
artemgavrilov May 16, 2023
d4427fc
PMM-7 Enable modules cache for QAN workflow
artemgavrilov May 16, 2023
d704b85
PMM-7 Fix main workflow
artemgavrilov May 16, 2023
26d9bae
PMM-7 Enable go build caches
artemgavrilov May 16, 2023
4fcfe5a
PMM-7 Fix
artemgavrilov May 16, 2023
f4f609a
PMM-7 Hit CI
artemgavrilov May 16, 2023
b332818
PMM-7 Add missing build cache for admin workflow
artemgavrilov May 17, 2023
2de336f
Merge branch 'main' into PMM-7-fix-ci-caches
artemgavrilov May 17, 2023
e0756e0
Merge branch 'main' into PMM-7-fix-ci-caches
artemgavrilov May 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 26 additions & 48 deletions .github/workflows/admin.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,25 +18,12 @@ on:
- "update/**"
- "vmproxy/**"

workflow_call:
inputs:
go-version:
description: An array of Go versions to be tested against, in a format of json string.
required: false
type: string
default: '["tip"]'

jobs:
test:
name: Tests
runs-on: ubuntu-22.04
strategy:
fail-fast: false
matrix:
go-version: ${{ fromJson(inputs.go-version || '["1.20.x"]') }}

env:
GO_VERSION: ${{ matrix.go-version }}

defaults:
run:
Expand All @@ -47,31 +34,29 @@ jobs:
uses: actions/checkout@v3

- name: Set up Go release
if: env.GO_VERSION != 'tip'
uses: actions/setup-go@v4
with:
go-version: ${{ env.GO_VERSION }}
go-version-file: ${{ github.workspace }}/go.mod
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fetches go version directly from go.mod file

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great choice!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super

cache: false

- name: Set up Go tip
if: env.GO_VERSION == 'tip'
run: |
git clone --depth=1 https://go.googlesource.com/go $HOME/gotip
cd $HOME/gotip/src
./make.bash
echo "GOROOT=$HOME/gotip" >> $GITHUB_ENV
echo "$HOME/gotip/bin" >> $GITHUB_PATH
echo "$GOBIN" >> $GITHUB_PATH
- name: Enable Go build cache
uses: actions/cache@v3
with:
path: ~/.cache/go-build
key: ${{ runner.os }}-go-build-${{ github.ref }}-${{ hashFiles('**') }}
restore-keys: |
${{ runner.os }}-go-build-${{ github.ref }}-
${{ runner.os }}-go-build-

- name: Enable Go modules cache
uses: actions/cache@v3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-modules-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-modules-
key: ${{ runner.os }}-go-modules-${{ hashFiles('**/go.sum') }}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is why we can't use embedded in setup-go action cache. Our cache uses all go.mod files in the project, while setup-go can use only one file. However this feature may be implemented soon: actions/setup-go#371

restore-keys: ${{ runner.os }}-go-modules-

- name: Download Go modules
run: go mod download
run: go mod download -x
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enables verbose mode (i.e. it will print what it downloads)


- name: Build and install
run: make install
Expand Down Expand Up @@ -100,11 +85,6 @@ jobs:
runs-on: ubuntu-22.04
strategy:
fail-fast: false
matrix:
go-version: ${{ fromJson(inputs.go-version || '["1.20.x"]') }}

env:
GO_VERSION: ${{ matrix.go-version }}

defaults:
run:
Expand All @@ -115,31 +95,29 @@ jobs:
uses: actions/checkout@v3

- name: Set up Go release
if: env.GO_VERSION != 'tip'
uses: actions/setup-go@v4
with:
go-version: ${{ env.GO_VERSION }}
go-version-file: ${{ github.workspace }}/go.mod
cache: false

- name: Set up Go tip
if: env.GO_VERSION == 'tip'
run: |
git clone --depth=1 https://go.googlesource.com/go $HOME/gotip
cd $HOME/gotip/src
./make.bash
echo "GOROOT=$HOME/gotip" >> $GITHUB_ENV
echo "$HOME/gotip/bin" >> $GITHUB_PATH
echo "$GOBIN" >> $GITHUB_PATH
- name: Enable Go build cache
uses: actions/cache@v3
with:
path: ~/.cache/go-build
key: ${{ runner.os }}-go-build-${{ github.ref }}-${{ hashFiles('**') }}
restore-keys: |
${{ runner.os }}-go-build-${{ github.ref }}-
${{ runner.os }}-go-build-

- name: Enable Go modules cache
uses: actions/cache@v3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-modules-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-modules-
key: ${{ runner.os }}-go-modules-${{ hashFiles('**/go.sum') }}
restore-keys: ${{ runner.os }}-go-modules-

- name: Download Go modules
run: go mod download
run: go mod download -x

- name: Build and install
run: make -C ../admin install
Expand Down
45 changes: 15 additions & 30 deletions .github/workflows/agent.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,23 +19,12 @@ on:
- "update/**"
- "vmproxy/**"

workflow_call:
inputs:
go-version:
description: An array of Go versions to be tested against, in a format of json string.
required: false
type: string
default: '["tip"]'

jobs:
test:
name: Tests
runs-on: ubuntu-22.04
strategy:
matrix:
go-version: ${{ fromJson(inputs.go-version || '["1.20.x"]') }}
may-fail: [ true ]

images:
- { mysql: 'mysql:5.6', mongo: 'mongo:4.2', postgres: 'postgres:10', pmm_server: 'percona/pmm-server:2.0.0' }
- { mysql: 'mysql:5.7', mongo: 'mongo:4.4', postgres: 'postgres:11', pmm_server: 'percona/pmm-server:2.0.1' }
Expand All @@ -59,10 +48,9 @@ jobs:
- { mysql: 'mariadb:10.3', mongo: 'percona/percona-server-mongodb:4.4', postgres: 'postgres:9.5', pmm_server: 'perconalab/pmm-server:dev-latest' }
- { mysql: 'mariadb:10.4', postgres: 'postgres:9.6', pmm_server: 'perconalab/pmm-server:dev-latest' }

continue-on-error: ${{ matrix.may-fail }}
continue-on-error: true

env:
GO_VERSION: ${{ matrix.go-version }}
MYSQL_IMAGE: ${{ matrix.images.mysql }}
MONGO_IMAGE: ${{ matrix.images.mongo }}
POSTGRES_IMAGE: ${{ matrix.images.postgres }}
Expand All @@ -78,32 +66,29 @@ jobs:
uses: actions/checkout@v3

- name: Set up Go release
if: env.GO_VERSION != 'tip'
uses: actions/setup-go@v4
with:
go-version: ${{ env.GO_VERSION }}
go-version-file: ${{ github.workspace }}/go.mod
cache: false

- name: Set up Go tip
if: env.GO_VERSION == 'tip'
run: |
git clone --depth=1 https://go.googlesource.com/go $HOME/gotip
cd $HOME/gotip/src
./make.bash
echo "GOROOT=$HOME/gotip" >> $GITHUB_ENV
echo "$HOME/gotip/bin" >> $GITHUB_PATH
- name: Enable Go build cache
uses: actions/cache@v3
with:
path: ~/.cache/go-build
key: ${{ runner.os }}-go-build-${{ github.ref }}-${{ hashFiles('**') }}
restore-keys: |
${{ runner.os }}-go-build-${{ github.ref }}-
${{ runner.os }}-go-build-

- name: Enable Go modules cache
uses: actions/cache@v3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-modules-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-modules-
key: ${{ runner.os }}-go-modules-${{ hashFiles('**/go.sum') }}
restore-keys: ${{ runner.os }}-go-modules-

- name: Download Go modules
run: |
pushd ../tools && go mod download
popd && go mod download
run: go mod download -x

- name: Build and install
run: make install
Expand All @@ -121,7 +106,7 @@ jobs:
with:
file: cover.out
flags: agent
env_vars: GO_VERSION,MYSQL_IMAGE,MONGO_IMAGE,POSTGRES_IMAGE,PMM_SERVER_IMAGE
env_vars: MYSQL_IMAGE,MONGO_IMAGE,POSTGRES_IMAGE,PMM_SERVER_IMAGE
fail_ci_if_error: false

- name: Run debug commands on failure
Expand Down
22 changes: 0 additions & 22 deletions .github/workflows/go-tip-checks.yml
View file
Open in desktop

This file was deleted.

44 changes: 15 additions & 29 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,58 +10,44 @@ on:

pull_request:

workflow_call:
inputs:
go-version:
description: An array of Go versions to be tested against, in a format of json string.
required: false
type: string
default: '["tip"]'

jobs:
check:
name: Checks
runs-on: ubuntu-22.04
strategy:
fail-fast: false
matrix:
go-version: ${{ fromJson(inputs.go-version || '["1.20.x"]') }}
may-fail: [ false ]

env:
GO_VERSION: ${{ matrix.go-version }}

steps:
- name: Check out code
uses: actions/checkout@v3

- name: Set up Go release
if: env.GO_VERSION != 'tip'
uses: actions/setup-go@v4
with:
go-version: ${{ env.GO_VERSION }}
go-version-file: ${{ github.workspace }}/go.mod
cache: false

- name: Set up Go tip
if: env.GO_VERSION == 'tip'
run: |
git clone --depth=1 https://go.googlesource.com/go $HOME/gotip
cd $HOME/gotip/src
./make.bash
echo "GOROOT=$HOME/gotip" >> $GITHUB_ENV
echo "$HOME/gotip/bin" >> $GITHUB_PATH
- name: Enable Go build cache
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build cache saves almost two minutes in this workflow

uses: actions/cache@v3
with:
path: ~/.cache/go-build
key: ${{ runner.os }}-go-build-${{ github.ref }}-${{ hashFiles('**') }}
restore-keys: |
${{ runner.os }}-go-build-${{ github.ref }}-
${{ runner.os }}-go-build-

- name: Enable Go modules cache
uses: actions/cache@v3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-modules-${{ hashFiles('**/go.sum') }}
key: ${{ runner.os }}-go-modules-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-modules-
${{ runner.os }}-go-modules-

- name: Download Go modules
run: |
pushd tools && go mod download
popd && go mod download
pushd tools && go mod download -x
popd && go mod download -x

- name: Install development tools
run: make init
Expand Down Expand Up @@ -98,7 +84,7 @@ jobs:
uses: reviewdog/action-golangci-lint@v2
with:
github_token: ${{ secrets.ROBOT_TOKEN || secrets.GITHUB_TOKEN }}
go_version: ${{ env.GO_VERSION }}
go_version_file: ${{ github.workspace }}/go.mod
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to setup-go. Reviewdog will fetch go version from go.mod file.

reporter: github-pr-review
fail_on_error: true
cache: false
Expand Down
29 changes: 8 additions & 21 deletions .github/workflows/managed.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,27 +22,15 @@ on:
- "update/**"
- "vmproxy/**"

workflow_call:
inputs:
go-version:
description: An array of Go versions to be tested against, in a format of json string.
required: false
type: string
default: '["tip"]'

jobs:
test:
name: Tests
runs-on: ubuntu-22.04
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
use-cache: [ false ]
go-version: ${{ fromJson(inputs.go-version || '["1.20.x"]') }}

env:
GO_VERSION: ${{ matrix.go-version }}
PMM_SERVER_IMAGE: perconalab/pmm-server:dev-latest
AWS_ACCESS_KEY: ${{ secrets.AWS_ACCESS_KEY }}
AWS_SECRET_KEY: ${{ secrets.AWS_SECRET_KEY }}
Expand All @@ -58,25 +46,24 @@ jobs:
uses: actions/cache@v3
with:
path: ~/.cache/go-build
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-build-${{ github.ref }}-${{ hashFiles('**') }}
key: ${{ runner.os }}-go-build-${{ github.ref }}-${{ hashFiles('**') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-build-${{ github.ref }}-
${{ runner.os }}-go-${{ env.GO_VERSION }}-build-
${{ runner.os }}-go-build-${{ github.ref }}-
${{ runner.os }}-go-build-

- name: Enable Go modules cache
if: matrix.use-cache
uses: actions/cache@v3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-modules-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-modules-
key: ${{ runner.os }}-go-modules-${{ hashFiles('**/go.sum') }}
restore-keys: ${{ runner.os }}-go-modules-

- name: Download Go modules
if: matrix.use-cache
run: |
pushd tools && go mod download
popd && go mod download
pushd tools && go mod download -x
popd && go mod download -x

- name: Initialize CI environment
run: make env-compose-up
Expand Down Expand Up @@ -105,7 +92,7 @@ jobs:
with:
file: managed/cover.out
flags: managed
env_vars: GO_VERSION,PMM_SERVER_IMAGE
env_vars: PMM_SERVER_IMAGE
fail_ci_if_error: false

- name: Cache
Expand Down
Loading