Skip to content

Commit

Permalink
Add compatibility tests to CI (open-telemetry#1567)
Browse files Browse the repository at this point in the history 
* Add compatibility tests to CI

* Increase timeout to 60s

Some of the aggregator tests are running just over 30 seconds when
testing on Windows. Bump this to allow for it.

* Remove cache setup jobs

Having a unified job setup the cache when none exists so packages are
only downloaded once is ideal, but these steps are taking too long. In
some situations they are doubling the run time. Remove them and prefer
to have multiple downloads when dependencies change.

* Add changes to changelog

* Remove mod-download target from Makefile

Unused now that the cache jobs are removed.

* Add compatibility testing table to changelog

* Add change about precommit to changelog

* Update README

* Correct examples target in lint job
  • Loading branch information
@MrAlias @ldelossa
MrAlias authored and ldelossa committed Mar 5, 2021
1 parent ed3afc7 commit 613ea74
Show file tree
Hide file tree
Showing 4 changed files with 265 additions and 137 deletions.
174 changes: 135 additions & 39 deletions .github/workflows/ci.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,46 +5,142 @@ on:
- main
pull_request:
env:
TEST_RESULTS: /tmp/test-results # path to where test results will be saved
# Path to where test results will be saved.
TEST_RESULTS: /tmp/test-results
# Default minimum version of Go to support.
DEFAULT_GO_VERSION: 1.14
jobs:
go:
lint:
runs-on: ubuntu-latest
steps:
- name: Install Go
uses: actions/setup-go@v2.1.3
with:
go-version: ${{ env.DEFAULT_GO_VERSION }}
- name: Checkout Repo
uses: actions/checkout@v2
- name: Setup Environment
run: |
echo "GOPATH=$(go env GOPATH)" >> $GITHUB_ENV
echo "$(go env GOPATH)/bin" >> $GITHUB_PATH
- name: Module cache
uses: actions/cache@v2.1.4
env:
cache-name: go-mod-cache
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-${{ env.cache-name }}-${{ hashFiles('**/go.sum') }}
- name: Tools cache
uses: actions/cache@v2.1.4
env:
cache-name: go-tools-cache
with:
path: ~/.tools
key: ${{ runner.os }}-${{ env.cache-name }}-${{ hashFiles('./internal/tools/**') }}
- name: Run linters
run: make dependabot-check license-check lint
- name: Build
run: make examples build
- name: Check clean repository
run: make check-clean-work-tree

test-race:
runs-on: ubuntu-latest
steps:
- name: Install Go
uses: actions/setup-go@v2.1.3
with:
go-version: ${{ env.DEFAULT_GO_VERSION }}
- name: Checkout Repo
uses: actions/checkout@v2
- name: Setup Environment
run: |
echo "GOPATH=$(go env GOPATH)" >> $GITHUB_ENV
echo "$(go env GOPATH)/bin" >> $GITHUB_PATH
- name: Module cache
uses: actions/cache@v2.1.4
env:
cache-name: go-mod-cache
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-${{ env.cache-name }}-${{ hashFiles('**/go.sum') }}
- name: Run tests with race detector
run: make test-race

test-coverage:
runs-on: ubuntu-latest
steps:
- name: Install Go
uses: actions/setup-go@v2.1.3
with:
go-version: ${{ env.DEFAULT_GO_VERSION }}
- name: Checkout Repo
uses: actions/checkout@v2
- name: Setup Environment
run: |
echo "GOPATH=$(go env GOPATH)" >> $GITHUB_ENV
echo "$(go env GOPATH)/bin" >> $GITHUB_PATH
- name: Module cache
uses: actions/cache@v2.1.4
env:
cache-name: go-mod-cache
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-${{ env.cache-name }}-${{ hashFiles('**/go.sum') }}
- name: Run coverage tests
run: |
make test-coverage
mkdir $TEST_RESULTS
cp coverage.out $TEST_RESULTS
cp coverage.txt $TEST_RESULTS
cp coverage.html $TEST_RESULTS
- name: Upload coverage report
uses: codecov/codecov-action@v1
with:
file: ./coverage.txt
fail_ci_if_error: true
verbose: true
- name: Store coverage test output
uses: actions/upload-artifact@v2
with:
name: opentelemetry-go-test-output
path: ${{ env.TEST_RESULTS }}

compatibility-test:
strategy:
fail-fast: false
matrix:
go_version: [1.14, 1.15]
runs-on: ubuntu-latest
go-version: [1.15, 1.14]
os: [ubuntu-latest, macos-latest, windows-latest]
# GitHub Actions does not support arm* architectures on default
# runners. It is possible to acomplish this with a self-hosted runner
# if we want to add this in the future:
# https://docs.github.com/en/actions/hosting-your-own-runners/using-self-hosted-runners-in-a-workflow
arch: ["386", amd64]
exclude:
# Not a supported Go OS/architecture.
- os: macos-latest
arch: "386"
runs-on: ${{ matrix.os }}
steps:
- name: Checkout Repo
uses: actions/checkout@v2
- name: Setup Go
uses: actions/setup-go@v2.1.3
with:
go-version: ${{ matrix.go_version }}
- name: Setup Environment
run: |
echo "GOPATH=$(go env GOPATH)" >> $GITHUB_ENV
echo "$(go env GOPATH)/bin" >> $GITHUB_PATH
- name: restore cache
uses: actions/cache@v2.1.4
env:
cache-name: cache-go-mod
with:
path: /home/runner/go/pkg/mod
key: go-pkg-mod-{{ checksum "go.sum" }}
- name: Precommit and Coverage Report
run: |
make ci
mkdir $TEST_RESULTS
cp coverage.out $TEST_RESULTS
cp coverage.txt $TEST_RESULTS
cp coverage.html $TEST_RESULTS
- uses: codecov/codecov-action@v1
with:
file: ./coverage.txt
fail_ci_if_error: true
verbose: true
- name: store test output
uses: actions/upload-artifact@v2
with:
name: opentelemetry-go-test-output
path: ${{ env.TEST_RESULTS }}
- name: Install Go
uses: actions/setup-go@v2.1.3
with:
go-version: ${{ matrix.go-version }}
- name: Checkout code
uses: actions/checkout@v2
- name: Setup Environment
run: |
echo "GOPATH=$(go env GOPATH)" >> $GITHUB_ENV
echo "$(go env GOPATH)/bin" >> $GITHUB_PATH
shell: bash
- name: Module cache
uses: actions/cache@v2.1.4
env:
cache-name: go-mod-cache
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-${{ env.cache-name }}-${{ hashFiles('**/go.sum') }}
- name: Run tests
env:
GOARCH: ${{ matrix.arch }}
run: make test-short
17 changes: 17 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,19 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
- AttributePerEventCountLimit and AttributePerLinkCountLimit for SpanLimits. (#1535)
- Added `Keys()` method to `propagation.TextMapCarrier` and `propagation.HeaderCarrier` to adapt `http.Header` to this interface. (#1544)
- Added `code` attributes to `go.opentelemetry.io/otel/semconv` package. (#1558)
- Compatibility testing suite in the CI system for the following systems. (#1567)
| OS | Go Version | Architecture |
| ------- | ---------- | ------------ |
| Ubuntu | 1.15 | amd64 |
| Ubuntu | 1.14 | amd64 |
| Ubuntu | 1.15 | 386 |
| Ubuntu | 1.14 | 386 |
| MacOS | 1.15 | amd64 |
| MacOS | 1.14 | amd64 |
| Windows | 1.15 | amd64 |
| Windows | 1.14 | amd64 |
| Windows | 1.15 | 386 |
| Windows | 1.14 | 386 |
- Changed all examples to use `WithBatchTimeout(5 * time.Second)` rather than `WithBatchTimeout(5)` (#1621)

### Changed
Expand All @@ -24,6 +37,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
- Rename MaxEventsPerSpan, MaxAttributesPerSpan and MaxLinksPerSpan to EventCountLimit, AttributeCountLimit and LinkCountLimit, and move these fieds into SpanLimits. (#1535)
- Renamed the `otel/label` package to `otel/attribute`. (#1541)
- Vendor the Jaeger exporter's dependency on Apache Thrift. (#1551)
- Parallelize the CI linting and testing. (#1567)
- Stagger timestamps in exact aggregator tests. (#1569)
- Prevent end-users from implementing some interfaces (#1575)
```
Expand All @@ -44,6 +58,9 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
### Removed

- Removed attempt to resample spans upon changing the span name with `span.SetName()`. (#1545)
- The `test-benchmark` is no longer a dependency of the `precommit` make target. (#1567)
- The `test-386` make target.
This was replaced with a full compatibility testing suite (i.e. multi OS/arch) in the CI system. (#1567)

### Fixed

Expand Down
Loading

0 comments on commit 613ea74

Please sign in to comment.