Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add end-to-end test execution to Continuous Integration #54

Merged
merged 1 commit into from
Dec 10, 2019
Merged

Add end-to-end test execution to Continuous Integration #54

merged 1 commit into from
Dec 10, 2019

Conversation

tigrannajaryan
Copy link
Member

@tigrannajaryan tigrannajaryan commented Dec 9, 2019 •
edited
Loading

@tigrannajaryan tigrannajaryan changed the title Add end-to-end test execution to Continuous Integration [WIP] Add end-to-end test execution to Continuous Integration Dec 9, 2019
@tigrannajaryan tigrannajaryan force-pushed the feature/tigran/enable-testbed-ci branch 3 times, most recently from 60de6f9 to deae7bd Compare December 9, 2019 19:10
@tigrannajaryan tigrannajaryan removed the WIP label Dec 9, 2019
@tigrannajaryan tigrannajaryan changed the title [WIP] Add end-to-end test execution to Continuous Integration Add end-to-end test execution to Continuous Integration Dec 9, 2019
Add end-to-end test execution to Continuous Integration
1693587 
- Now end-to-end tests from the Testbed are executed as part of CirleCI build.
- Test summary are published in CircleCI using JUnit format.
- All test result files (e.g. logs) are published as CircleCO artifacts.
pjanotti
pjanotti approved these changes Dec 10, 2019
View reviewed changes
Copy link
Contributor

@pjanotti pjanotti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pjanotti pjanotti merged commit ff96a11 into open-telemetry:master Dec 10, 2019
@tigrannajaryan tigrannajaryan deleted the feature/tigran/enable-testbed-ci branch December 10, 2019 16:49
mxiamxia referenced this pull request in mxiamxia/opentelemetry-collector-contrib Jul 22, 2020
@tigrannajaryan
Add End-to-end test bed (#54)
b4c9671 
Problem
=======

We need a way to run E2E tests with real agent and a load generator. Particularly
important class of E2E tests are performance tests in controlled environment.

Solution
========

The test bed allows to easily set up a test that requires running the agent
and a load generator, measure and define resource consumption expectations
for the agent, fail tests automatically when expectations are exceeded.

Each test case requires a agent configuration file and (optionally) load
generator spec file. Test cases are defined as regular Go tests, see examples
in perf_test.go.

To run E2E tests go to testbed directory and run "make runtests" or run "make perf-tests"
from top-level directory.

Note that E2E tests are skipped when running regular "make test" target. The reason
is that because E2E tests take a lot of time and require non-concurrent execution
while other tests are typical run with -race flag.

See also design doc here:
https://docs.google.com/document/d/1omU06mBYGY0slT1yojttn9BCyp18pHaRjkkYZrY8H4Q/edit#

TODO
=====

We plan to add the following functionality later:

- Ability for load generator to record what it generated and then use that
  information to validate data received by mock backend from the agent.

- Enforce resource consumption limits on the agent to allow testing scenarios under
  specific available resource situations (as opposed to measuring and failing tests
  which is already implemented and which is useful in different scenarios).
ljmsc referenced this pull request in ljmsc/opentelemetry-collector-contrib Feb 21, 2022
@rghetia
replace import with vanity url go.opentelemetry.io. (#54)
eed647d
bogdandrutu pushed a commit that referenced this pull request May 12, 2022
@djaglowski
Update changelog ahead of v0.16.0 release (#54)
0c7c5c5 
* Update changelog ahead of v0.16.0 release
hex1848 pushed a commit to hex1848/opentelemetry-collector-contrib that referenced this pull request Jun 2, 2022
@himanshu007-creator
Update to collector v0.29.0 (open-telemetry#54)
9225b86 
* Update replaces.builder.yaml

* Update nocore.builder.yaml

* Update config.go

* Update README.md

* Update main.go
hex1848 pushed a commit to hex1848/opentelemetry-collector-contrib that referenced this pull request Jun 2, 2022
@jpkrohling @bogdandrutu
@joe-elliott @jiwen624 @Ashmita152 @fangyi-zhou @crearys @pmalek-sumo @secat @sodabrew @pmalek @KKelvinLo @himanshu007-creator @u5surf @zchee @Aneurysm9 @loomis-relativity
Move opentelemetry-collector-builder to core (open-telemetry#4286)
4063a46 
* Initial commit

* Add CODEOWNERS file (open-telemetry#2)

* Add CODEOWNERS file

* Update CODEOWNERS

* Moved from github.com/observatorium/opentelemetry-collector-builder (open-telemetry#3)

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* fixed panics (open-telemetry#6)

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Replace master with main in CI and mergify files (open-telemetry#8)

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Bump to OpenTelemetry Collector 0.20.0 (open-telemetry#10)

Closes open-telemetry#9

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Explicitly enable Go modules in quickstart instructions (open-telemetry#13)

* Update to collector v0.21.0 (open-telemetry#17)

Fixes open-telemetry#16

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Update to collector v0.22.0 (open-telemetry#19)

* Download go modules before building (open-telemetry#20)

Fixes open-telemetry#14

* Add version command (open-telemetry#25)

Signed-off-by: Ashmita Bohara <ashmita.bohara152@gmail.com>

* Pass errors from cobra Execute back to main for correct exit code (open-telemetry#28)

* pass errors from cobra execute back to main

* print the error

* Update to collector v0.23.0 (open-telemetry#27)

* Generate a warning if the builder and collector base version mismatch (open-telemetry#30)

* Generate a warning if the builder and collector base version mismatch

* Show current default version in the warning message

* Update to OpenTelemetry Collector 0.24.0

* Don't use %w formatting with log.Fatal (open-telemetry#35)

* Update to OpenTelemetry Collector 0.25.0 (open-telemetry#36)

Signed-off-by: Serge Catudal <serge.catudal@gmail.com>

* Update to 0.26.0 and update BuildInfo (open-telemetry#39)

* Sync build and CI Go versions at latest 1.16 (open-telemetry#34)

* Sync build and CI Go versions at latest 1.16

* Run go mod tidy

* Set go binary to use in the compilation phase in tests

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

Co-authored-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Add option to generate go code only (no compile) (open-telemetry#40)

* Issue#24 Add option to generate go code only (no compile)

* Update cmd/root.go logging

Suggested by @jpkkrohling

Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de>

* remove verbose help .. created by corba

* suggestion by jpkrohling to keep generateandcompile

* lint error: remove unused var

* reword cmd option and add back help message for default

Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de>

* Don't reuse exec.Cmd (open-telemetry#42)

* Update to OpenTelemetry Collector 0.27.0 (open-telemetry#43)

* Add CI Badge (open-telemetry#47)

* Update to Collector v0.28.0 (open-telemetry#49)

* Update to Collector v0.28.0

Closes open-telemetry#48

Addresses the breaking API change in
open-telemetry/opentelemetry-collector#3163,
besides the usual version number changes.

Signed-off-by: Fangyi Zhou <me@fangyi.io>

* Use `go mod tidy` instead of `go mod download`

It appears that this magically resolves the go.mod file issue.
https://stackoverflow.com/questions/67203641/missing-go-sum-entry-for-module-providing-package-package-name

Signed-off-by: Fangyi Zhou <me@fangyi.io>

* Account for go mod download in go1.17 not updating go.sum (open-telemetry#50)

* Update to collector v0.29.0 (open-telemetry#54)

* Update replaces.builder.yaml

* Update nocore.builder.yaml

* Update config.go

* Update README.md

* Update main.go

* Update to collector v0.30.0 (open-telemetry#57)

* cmd: fix module flag default value to github.com/open-telemetry (open-telemetry#58)

Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>

* Update to collector v0.31.0 (open-telemetry#60)

* Update to v0.33.0 (open-telemetry#62)

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* Add excludes support to generated go.mod (open-telemetry#63)

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

Co-authored-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Small cleanup for the builder files (open-telemetry#64)

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

* Support building with Go 1.17 (open-telemetry#66)

* Support building with Go 1.17
Fixes open-telemetry#65

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Update workflows to use Go 1.17

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Add gosec exceptions for exec.Command

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Update to OpenTelemetry core 0.34.0 (open-telemetry#68)

Fixes open-telemetry#67

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Upgrade to OpenTelemetry Collector 0.35.0 (open-telemetry#70)

Signed-off-by: Fangyi Zhou <me@fangyi.io>

* Upgrade to OpenTelemetry Collector 0.36.0 (open-telemetry#76)

* Generate custom service code for Windows (open-telemetry#75)

* update main to include windows service code

* use main version from tag 0.35.0

* update main function

* align with upstream v0.36.0 tag

* dummy change to trigger build

* Revert "dummy change to trigger build"

This reverts commit 629d499461da2d2c240bf1e495b5fe0558e3547f.

* Remove Core from Module type (open-telemetry#77)

Fixes open-telemetry#15

Signed-off-by: yugo-horie <u5.horie@gmail.com>

* release 0.37.0 (open-telemetry#78)

* release 0.37.0

* update use of NewCommand

* Move builder to subdirectory

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

Co-authored-by: Bogdan Drutu <lazy@splunk.com>
Co-authored-by: Bogdan Drutu <bogdandrutu@gmail.com>
Co-authored-by: Joe Elliott <joe.elliott@grafana.com>
Co-authored-by: Eric Yang <jiwen624@gmail.com>
Co-authored-by: Brian Gibbins <eroteme@supernought.co.uk>
Co-authored-by: Ashmita <ashmita.bohara152@gmail.com>
Co-authored-by: Fangyi Zhou <me@fangyi.io>
Co-authored-by: Shaun Creary <65406540+crearys@users.noreply.github.com>
Co-authored-by: Patryk Małek <69143962+pmalek-sumo@users.noreply.github.com>
Co-authored-by: Serge Catudal <serge.catudal@gmail.com>
Co-authored-by: Aaron Stone <aaron@serendipity.cx>
Co-authored-by: Patryk Małek <pmalek@sumologic.com>
Co-authored-by: Aaron Stone <aaron.stone@udacity.com>
Co-authored-by: Kelvin Lo <kello@live.ca>
Co-authored-by: Himanshu <addyjeridiq@gmail.com>
Co-authored-by: Y.Horie <u5.horie@gmail.com>
Co-authored-by: Koichi Shiraishi <zchee.io@gmail.com>
Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>
Co-authored-by: Cal Loomis <68860480+loomis-relativity@users.noreply.github.com>
Co-authored-by: alrex <aboten@lightstep.com>
jj22ee pushed a commit to jj22ee/opentelemetry-collector-contrib that referenced this pull request Sep 21, 2023
@sethAmazon
Merge pull request open-telemetry#54 from sethAmazon/fix-vul-check
7bbc8b5 
Fix Golang Vulcheck
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pjanotti pjanotti pjanotti approved these changes

@owais owais owais approved these changes

@bogdandrutu bogdandrutu Awaiting requested review from bogdandrutu

@flands flands Awaiting requested review from flands

@rghetia rghetia Awaiting requested review from rghetia

@songy23 songy23 Awaiting requested review from songy23

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@tigrannajaryan @owais @pjanotti