Skip to content
This repository has been archived by the owner on Feb 5, 2020. It is now read-only.

Jenkins: CI test using a flake-resistent Terraform build #1247

Merged
merged 1 commit into from
Jul 7, 2017
Merged

Jenkins: CI test using a flake-resistent Terraform build #1247

merged 1 commit into from
Jul 7, 2017

Conversation

s-urbaniak
Copy link
Contributor

@s-urbaniak s-urbaniak commented Jun 29, 2017 •
edited
Loading

This is using a more flake-resistent Terraform build.

@s-urbaniak s-urbaniak mentioned this pull request Jun 29, 2017
Closed
@s-urbaniak s-urbaniak force-pushed the tf-flakes branch 3 times, most recently from dec9c38 to de48016 Compare July 4, 2017 14:38
@s-urbaniak s-urbaniak changed the title [WIP] Jenkins: CI test using a flake-resistent Terraform build Jenkins: CI test using a flake-resistent Terraform build Jul 4, 2017
@coreos coreos deleted a comment from sym3tri Jul 4, 2017
@coreos coreos deleted a comment from sym3tri Jul 4, 2017
@coreos coreos deleted a comment from sym3tri Jul 4, 2017
@coreos coreos deleted a comment from sym3tri Jul 4, 2017
@coreos coreos deleted a comment from sym3tri Jul 4, 2017
@s-urbaniak
Copy link
Contributor Author

/cc @mxinden @Quentin-M

mxinden
mxinden previously approved these changes Jul 5, 2017
View reviewed changes
@squat
Copy link
Contributor

squat commented Jul 5, 2017

re-running the tests as it looks like this PR is flaking quite a bit itself.

@s-urbaniak
Copy link
Contributor Author

ack, the flake hunt is definitely not over. This one happens now in the Go based integration tests.

@squat
Copy link
Contributor

squat commented Jul 6, 2017 •
edited
Loading

@s-urbaniak hmm it's failing occasionally in the killApiServer test. It looks like the test timed out. The API server came back eventually but was not up long enough for the test to be considered successful. This kind of flakiness is actually representative of the robustness of Tectonic components, and not of the robustness of the tests and certainly not of the PR. We can only really fix this in three ways:

  1. Redesigning the tests to not kill the apiserver or to measure success by some other criteria;
  2. Increase timeouts for this test and be satisfied with the fact that apiserver can take long to come up (special case of redefining success criteria); or
  3. Fix our checkpointer somehow so the apiserver comes up faster.

mxinden
mxinden approved these changes Jul 6, 2017
View reviewed changes
Copy link
Contributor

@mxinden mxinden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the comments above are not directly related to the changes here I would suggest merging this PR and taking care of the suggestions above in a separate PR.

@s-urbaniak @squat What do you think?

@s-urbaniak
Copy link
Contributor Author

I'd say let's merge this and we can hunt for other flakes in a follow-up PR. Also I predict that we will have to bump the custom TF binary from time to time to reflect the current state of upstream review comments/discussions.

@mxinden mxinden merged commit 0a26934 into coreos:master Jul 7, 2017
mxinden added a commit to mxinden/tectonic-installer that referenced this pull request Jul 20, 2017
@mxinden
tests: Add upstream terraform trigger
139feae 
We are currently using a custom terraform version for our default CI
pipeline (See coreos#1247).
As end users of Tectonic installer use upstream terraform we need to
test with upstream terraform as well. We are building and publishing the
Tectonic builder image both with CoreOS Terraform as well as upstream
Terraform. This is done via docker `--build-arg` `TERRAFORM_URL`. CoreOS
Terraform is used for normal PR and branch tests, upstream Terraform is
used once per day on master.

* Add `--build-arg` `TERRAFORM_URL` to Tectonic builder Docker file
* Update documentation in tectonic-builder/README.md
* Add upstream-terraform-trigger.groovy pipeline job

With this commit v1.34 and v1.34-upstream-terraform of the Tectonic
builder images are pushed to quay.io/coreos/tectonic-builder.
@mxinden mxinden mentioned this pull request Jul 20, 2017
Merged
mxinden added a commit to mxinden/tectonic-installer that referenced this pull request Jul 20, 2017
@mxinden
tests: Add upstream terraform trigger
069f63f 
We are currently using a custom terraform version for our default CI
pipeline (See coreos#1247).
As end users of Tectonic installer use upstream terraform we need to
test with upstream terraform as well. We are building and publishing the
Tectonic builder image both with CoreOS Terraform as well as upstream
Terraform. This is done via docker `--build-arg` `TERRAFORM_URL`. CoreOS
Terraform is used for normal PR and branch tests, upstream Terraform is
used once per day on master.

* Add `--build-arg` `TERRAFORM_URL` to Tectonic builder Docker file
* Update documentation in tectonic-builder/README.md
* Add upstream-terraform-trigger.groovy pipeline job

With this commit v1.34 and v1.34-upstream-terraform of the Tectonic
builder images are pushed to quay.io/coreos/tectonic-builder.
mxinden added a commit to mxinden/tectonic-installer that referenced this pull request Jul 20, 2017
@mxinden
tests: Add upstream terraform trigger
dbc55e2 
We are currently using a custom terraform version for our default CI
pipeline (See coreos#1247).
As end users of Tectonic installer use upstream terraform we need to
test with upstream terraform as well. We are building and publishing the
Tectonic builder image both with CoreOS Terraform as well as upstream
Terraform. This is done via docker `--build-arg` `TERRAFORM_URL`. CoreOS
Terraform is used for normal PR and branch tests, upstream Terraform is
used once per day on master.

* Add `--build-arg` `TERRAFORM_URL` to Tectonic builder Docker file
* Update documentation in tectonic-builder/README.md
* Add upstream-terraform-trigger.groovy pipeline job

With this commit v1.34 and v1.34-upstream-terraform of the Tectonic
builder images are pushed to quay.io/coreos/tectonic-builder.
mxinden added a commit to mxinden/tectonic-installer that referenced this pull request Jul 20, 2017
@mxinden
tests: Add upstream terraform trigger
5931e1a 
We are currently using a custom terraform version for our default CI
pipeline (See coreos#1247).
As end users of Tectonic installer use upstream terraform we need to
test with upstream terraform as well. We are building and publishing the
Tectonic builder image both with CoreOS Terraform as well as upstream
Terraform. This is done via docker `--build-arg` `TERRAFORM_URL`. CoreOS
Terraform is used for normal PR and branch tests, upstream Terraform is
used once per day on master.

* Add `--build-arg` `TERRAFORM_URL` to Tectonic builder Docker file
* Update documentation in tectonic-builder/README.md
* Add upstream-terraform-trigger.groovy pipeline job

With this commit v1.34 and v1.34-upstream-terraform of the Tectonic
builder images are pushed to quay.io/coreos/tectonic-builder.
mxinden added a commit to mxinden/tectonic-installer that referenced this pull request Jul 21, 2017
@mxinden
tests: Add upstream terraform trigger
9b3ddec 
We are currently using a custom terraform version for our default CI
pipeline (See coreos#1247).
As end users of Tectonic installer use upstream terraform we need to
test with upstream terraform as well. We are building and publishing the
Tectonic builder image both with CoreOS Terraform as well as upstream
Terraform. This is done via docker `--build-arg` `TERRAFORM_URL`. CoreOS
Terraform is used for normal PR and branch tests, upstream Terraform is
used once per day on master.

* Add `--build-arg` `TERRAFORM_URL` to Tectonic builder Docker file
* Update documentation in tectonic-builder/README.md
* Add upstream-terraform-trigger.groovy pipeline job

With this commit v1.36 and v1.36-upstream-terraform of the Tectonic
builder images are pushed to quay.io/coreos/tectonic-builder. There
are no functional changes, but changes to the build order in the
Dockerfile.

You can find the Jenkins job
[here](https://jenkins-tectonic-installer.prod.coreos.systems/job/upstream-terraform-trigger/).
mxinden added a commit to mxinden/tectonic-installer that referenced this pull request Jul 21, 2017
@mxinden
tests: Add upstream terraform trigger
323b380 
We are currently using a custom terraform version for our default CI
pipeline (See coreos#1247).
As end users of Tectonic installer use upstream terraform we need to
test with upstream terraform as well. We are building and publishing the
Tectonic builder image both with CoreOS Terraform as well as upstream
Terraform. This is done via docker `--build-arg` `TERRAFORM_URL`. CoreOS
Terraform is used for normal PR and branch tests, upstream Terraform is
used once per day on master.

* Add `--build-arg` `TERRAFORM_URL` to Tectonic builder Docker file
* Update documentation in tectonic-builder/README.md
* Add upstream-terraform-trigger.groovy pipeline job

With this commit v1.36 and v1.36-upstream-terraform of the Tectonic
builder images are pushed to quay.io/coreos/tectonic-builder. There
are no functional changes, but changes to the build order in the
Dockerfile.

You can find the Jenkins job
[here](https://jenkins-tectonic-installer.prod.coreos.systems/job/upstream-terraform-trigger/).
mxinden added a commit to mxinden/tectonic-installer that referenced this pull request Jul 21, 2017
@mxinden
tests: Add upstream terraform trigger
82a60f6 
We are currently using a custom terraform version for our default CI
pipeline (See coreos#1247).
As end users of Tectonic installer use upstream terraform we need to
test with upstream terraform as well. We are building and publishing the
Tectonic builder image both with CoreOS Terraform as well as upstream
Terraform. This is done via docker `--build-arg` `TERRAFORM_URL`. CoreOS
Terraform is used for normal PR and branch tests, upstream Terraform is
used once per day on master.

* Add `--build-arg` `TERRAFORM_URL` to Tectonic builder Docker file
* Update documentation in tectonic-builder/README.md
* Add upstream-terraform-trigger.groovy pipeline job

With this commit v1.36 and v1.36-upstream-terraform of the Tectonic
builder images are pushed to quay.io/coreos/tectonic-builder. There
are no functional changes, but changes to the build order in the
Dockerfile.

You can find the Jenkins job
[here](https://jenkins-tectonic-installer.prod.coreos.systems/job/upstream-terraform-trigger/).
mxinden added a commit to mxinden/tectonic-installer that referenced this pull request Jul 26, 2017
@mxinden
tests: Add upstream terraform trigger
7bc232b 
We are currently using a custom terraform version for our default CI
pipeline (See coreos#1247).
As end users of Tectonic installer use upstream terraform we need to
test with upstream terraform as well. We are building and publishing the
Tectonic builder image both with CoreOS Terraform as well as upstream
Terraform. This is done via docker `--build-arg` `TERRAFORM_URL`. CoreOS
Terraform is used for normal PR and branch tests, upstream Terraform is
used once per day on master.

* Add `--build-arg` `TERRAFORM_URL` to Tectonic builder Docker file
* Update documentation in tectonic-builder/README.md
* Add upstream-terraform-trigger.groovy pipeline job

With this commit v1.36 and v1.36-upstream-terraform of the Tectonic
builder images are pushed to quay.io/coreos/tectonic-builder. There
are no functional changes, but changes to the build order in the
Dockerfile.

You can find the Jenkins job
[here](https://jenkins-tectonic-installer.prod.coreos.systems/job/upstream-terraform-trigger/).
mxinden added a commit to mxinden/tectonic-installer that referenced this pull request Aug 2, 2017
@mxinden
tests: Add upstream terraform trigger
044d20f 
We are currently using a custom terraform version for our default CI
pipeline (See coreos#1247).
As end users of Tectonic installer use upstream terraform we need to
test with upstream terraform as well. We are building and publishing the
Tectonic builder image both with CoreOS Terraform as well as upstream
Terraform. This is done via docker `--build-arg` `TERRAFORM_URL`. CoreOS
Terraform is used for normal PR and branch tests, upstream Terraform is
used once per day on master.

* Add `--build-arg` `TERRAFORM_URL` to Tectonic builder Docker file
* Update documentation in tectonic-builder/README.md
* Add upstream-terraform-trigger.groovy pipeline job

With this commit v1.36 and v1.36-upstream-terraform of the Tectonic
builder images are pushed to quay.io/coreos/tectonic-builder. There
are no functional changes, but changes to the build order in the
Dockerfile.

You can find the Jenkins job
[here](https://jenkins-tectonic-installer.prod.coreos.systems/job/upstream-terraform-trigger/).
mxinden added a commit to mxinden/tectonic-installer that referenced this pull request Aug 9, 2017
@mxinden
tests: Add upstream terraform trigger
4659a6c 
We are currently using a custom terraform version for our default CI
pipeline (See coreos#1247).
As end users of Tectonic installer use upstream terraform we need to
test with upstream terraform as well. We are building and publishing the
Tectonic builder image both with CoreOS Terraform as well as upstream
Terraform. This is done via docker `--build-arg` `TERRAFORM_URL`. CoreOS
Terraform is used for normal PR and branch tests, upstream Terraform is
used once per day on master.

* Add `--build-arg` `TERRAFORM_URL` to Tectonic builder Docker file
* Update documentation in tectonic-builder/README.md
* Add upstream-terraform-trigger.groovy pipeline job

With this commit v1.36 and v1.36-upstream-terraform of the Tectonic
builder images are pushed to quay.io/coreos/tectonic-builder. There
are no functional changes, but changes to the build order in the
Dockerfile.

You can find the Jenkins job
[here](https://jenkins-tectonic-installer.prod.coreos.systems/job/upstream-terraform-trigger/).
mxinden added a commit to mxinden/tectonic-installer that referenced this pull request Aug 9, 2017
@mxinden
tests: Add upstream terraform trigger
15394f8 
We are currently using a custom terraform version for our default CI
pipeline (See coreos#1247).
As end users of Tectonic installer use upstream terraform we need to
test with upstream terraform as well. We are building and publishing the
Tectonic builder image both with CoreOS Terraform as well as upstream
Terraform. This is done via docker `--build-arg` `TERRAFORM_URL`. CoreOS
Terraform is used for normal PR and branch tests, upstream Terraform is
used once per day on master.

* Add `--build-arg` `TERRAFORM_URL` to Tectonic builder Docker file
* Update documentation in tectonic-builder/README.md
* Add upstream-terraform-trigger.groovy pipeline job

With this commit v1.36 and v1.36-upstream-terraform of the Tectonic
builder images are pushed to quay.io/coreos/tectonic-builder. There
are no functional changes, but changes to the build order in the
Dockerfile.

You can find the Jenkins job
[here](https://jenkins-tectonic-installer.prod.coreos.systems/job/upstream-terraform-trigger/).
squat pushed a commit to squat/tectonic-installer that referenced this pull request Sep 25, 2017
@mxinden @squat
tests: Add upstream terraform trigger
5f763fb 
We are currently using a custom terraform version for our default CI
pipeline (See coreos#1247).
As end users of Tectonic installer use upstream terraform we need to
test with upstream terraform as well. We are building and publishing the
Tectonic builder image both with CoreOS Terraform as well as upstream
Terraform. This is done via docker `--build-arg` `TERRAFORM_URL`. CoreOS
Terraform is used for normal PR and branch tests, upstream Terraform is
used once per day on master.

* Add `--build-arg` `TERRAFORM_URL` to Tectonic builder Docker file
* Update documentation in tectonic-builder/README.md
* Add upstream-terraform-trigger.groovy pipeline job

With this commit v1.36 and v1.36-upstream-terraform of the Tectonic
builder images are pushed to quay.io/coreos/tectonic-builder. There
are no functional changes, but changes to the build order in the
Dockerfile.

You can find the Jenkins job
[here](https://jenkins-tectonic-installer.prod.coreos.systems/job/upstream-terraform-trigger/).
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@mxinden mxinden mxinden approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@s-urbaniak @squat @mxinden