Skip to content

Commit

Permalink
ci-operator/templates/openshift: Drop us-east-1c
Browse files Browse the repository at this point in the history
We're currently hitting a lot of these.  Over the past 24 hours [1]:

  $ curl -s 'http://localhost:8000/search?name=-e2e-aws&.&q=level%3Derror.*timeout+while+waiting+for+state' | jq -r '. | to_entries[].value[] | to_entries[].value[]' | sed 's/(i-[^)]*/(i-.../;s/(igw-[^)]*/(igw-.../;s/\(master\|nat_gw\|private_routing\|route_net\)\.[0-9]/\1.../' | sort | uniq -c | sort -n
       2 level=error msg="\t* aws_instance.master...: Error waiting for instance (i-...) to become ready: timeout while waiting for state
      10 level=error msg="\t* aws_security_group.bootstrap: timeout while waiting for state
      38 level=error msg="\t* aws_route.igw_route: Error creating route: timeout while waiting for state
      58 level=error msg="\t* aws_internet_gateway.igw: error attaching EC2 Internet Gateway (igw-...): timeout while waiting for state
      76 level=error msg="\t* aws_route_table_association.private_routing...: timeout while waiting for state
      90 level=error msg="\t* aws_route_table_association.route_net...: timeout while waiting for state
     164 level=error msg="\t* aws_route.to_nat_gw...: Error creating route: timeout while waiting for state

Dropping to two zones will reduce our API load for the per-subnet
to_nat_gw routes and route-table associations, which are our leading
breakages.

Generated with:

  $ sed -i '/us-east-1c/d' $(git grep -l us-east-1c)

[1]: https://github.com/wking/openshift-release/tree/debug-scripts/d3
  • Loading branch information
wking committed Apr 26, 2019
1 parent 298e578 commit d87fffb
Show file tree
Hide file tree
Showing 4 changed files with 0 additions and 8 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,6 @@ objects:
zones:
- us-east-1a
- us-east-1b
- us-east-1c
compute:
- name: worker
replicas: 3
Expand All @@ -250,7 +249,6 @@ objects:
zones:
- us-east-1a
- us-east-1b
- us-east-1c
networking:
clusterNetwork:
- cidr: 10.128.0.0/14
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,6 @@ objects:
zones:
- us-east-1a
- us-east-1b
- us-east-1c
compute:
- name: worker
replicas: 3
Expand All @@ -252,7 +251,6 @@ objects:
zones:
- us-east-1a
- us-east-1b
- us-east-1c
networking:
clusterNetwork:
- cidr: 10.128.0.0/14
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,6 @@ objects:
zones:
- us-east-1a
- us-east-1b
- us-east-1c
compute:
- name: worker
replicas: 3
Expand All @@ -246,7 +245,6 @@ objects:
zones:
- us-east-1a
- us-east-1b
- us-east-1c
networking:
clusterNetwork:
- cidr: 10.128.0.0/14
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -328,7 +328,6 @@ objects:
zones:
- us-east-1a
- us-east-1b
- us-east-1c
compute:
- name: worker
replicas: 3
Expand All @@ -337,7 +336,6 @@ objects:
zones:
- us-east-1a
- us-east-1b
- us-east-1c
networking:
clusterNetwork:
- cidr: 10.128.0.0/14
Expand Down

0 comments on commit d87fffb

Please sign in to comment.