mkpjpod demos #2928

bbguimaraes · 2019-02-22T12:16:36Z

The test-infra:binaries image still needs to be fixed before the new, two-argument invocation of mkpjpod can be used. I also reworked the (embarrassing) handling of docker command line arguments.

As for the demos, these are the first two, covering the basics. I will update this PR as I record the remaining. Feedback is appreciated — it is easy for me to re-record them (when api.ci is happy, that is).

droslean · 2019-02-22T14:01:00Z

Shall we have a script to run the .cast file?

bbguimaraes · 2019-02-22T17:58:08Z

@droslean: added a mention of the recording/playback application in the documentation.

droslean · 2019-02-25T08:56:44Z

There is also an asciinema image. Do you think that we can use that to run the .cast file directly?

bbguimaraes · 2019-02-25T10:33:58Z

We could upload it to asciinema.org, it is all public information anyway.

bbguimaraes · 2019-02-25T16:26:52Z

/test ci-operator-config

stevekuznetsov · 2019-02-25T18:49:06Z

I've seen these embedded into READMEs -- can we do that here?

bbguimaraes · 2019-02-25T20:21:01Z

Kind of:

bbguimaraes · 2019-02-28T13:59:32Z

!?

bbguimaraes · 2019-02-28T14:01:17Z

Seems to be failing in all PRs:

[ERROR] The following errors were found:

diff -Naupr hack/../ci-operator/jobs/openshift/openshift-ansible/openshift-openshift-ansible-devel-40-presubmits.yaml /tmp/tmp.VCUiSe6H43/ci-operator/jobs/openshift/openshift-ansible/openshift-openshift-ansible-devel-40-presubmits.yaml
--- hack/../ci-operator/jobs/openshift/openshift-ansible/openshift-openshift-ansible-devel-40-presubmits.yaml	2019-02-28 13:53:57.159247554 +0000
+++ /tmp/tmp.VCUiSe6H43/ci-operator/jobs/openshift/openshift-ansible/openshift-openshift-ansible-devel-40-presubmits.yaml	2019-02-28 13:54:09.848355150 +0000
@@ -106,7 +106,7 @@ presubmits:
           name: cluster-profile
         - mountPath: /usr/local/e2e-gcp
           name: job-definition
-          subPath: cluster-launch-e2e-40.yaml
+          subPath: cluster-scaleup-e2e-40.yaml
       serviceAccountName: ci-operator
       volumes:
       - name: cluster-profile
@@ -117,7 +117,7 @@ presubmits:
           - configMap:
               name: cluster-profile-gcp-40
       - configMap:
-          name: prow-job-cluster-launch-e2e-40
+          name: prow-job-cluster-scaleup-e2e-40
         name: job-definition
     trigger: '(?m)^/test (?:.*? )?e2e-gcp(?: .*?)?$'
   - agent: kubernetes

droslean · 2019-02-28T14:13:06Z

/test generated-config

stevekuznetsov · 2019-02-28T16:00:30Z

@bbguimaraes what's left on this one?

bbguimaraes · 2019-03-01T21:02:22Z

Added a demo that makes changes to the ci-operator configuration file.

asciinema.org requires an account to store recordings. It wouldn't be a detriment, I can create the account and put the credentials in BitWarden.

There doesn't seem to be any convenient way to embed recordings otherwise.

droslean · 2019-03-01T21:05:39Z

I am fine with that.

bbguimaraes · 2019-03-01T21:31:19Z

And because they have such an aggressive pruning policy, I feel less bad putting WIP recordings there:

mkpjpod.cast

mkpjpod_modify_jobs.cast

droslean · 2019-03-01T21:38:07Z

The explanation you are writing in the cast goes out of the screen and the whole sentence is not readable. Can you check it?

bbguimaraes · 2019-03-04T12:39:32Z

@droslean: do you mean text doesn't wrap when it is supposed to, e.g. around the 0:15 mark on the first demo? I tested that in different scenarios (small and big terminal sizes, asciinema.org's player) and it displays fine.

bbguimaraes · 2019-03-04T12:39:37Z

I've set up the account: https://asciinema.org/~origin-ci. The casts are already there, but private for now.

The login process doesn't use passwords, it sends an email with a token that is valid for 15min. I used my @redhat email for now, We have a team mailing list that can be used for that, right? I will change the email as soon as I remember what it is =)

Also, it generated a random profile picture but uses gravatar.com, if we care.

If we think the demos are good as they are, I'll publish them and update the documentation. I don't think we will need the .cast files in openshift/release, as they can be downloaded from asciinema.org.

stevekuznetsov · 2019-03-04T16:08:24Z

All of that SGTM @bbguimaraes

bbguimaraes · 2019-03-05T14:16:22Z

account email set
casts titled and published (see here)
documentation updated (see here)
rebased

droslean · 2019-03-05T14:26:31Z

/lgtm

openshift-ci-robot · 2019-03-05T14:26:40Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bbguimaraes, droslean

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [bbguimaraes,droslean]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…ions Catching up with b717933 (ci-operator/templates/openshift/installer/cluster-launch-installer-*: Random AWS regions for IPI, 2020-01-23, openshift#6833), add shares subnets for the new regions. This is basically as described in 424a04a (Add shared vpc CI job for installer repo and informing periodics, 2019-08-22, openshift#5550), although I've dropped the --stack-policy-body mentioned in that commit message (I dunno what it was about, since the commit message didn't give it the argument that --stack-policy-body expects). Generated using: $ export AWS_PROFILE=ci # or whatever you call it locally $ git fetch origin $ date --iso=m --utc 2020-01-23T20:41+0000 $ git checkout origin/release-4.3 $ git --no-pager log --oneline -1 2055609f9 (HEAD, origin/release-4.3) Merge pull request openshift#2928 from ashcrow/4.3-signed-rhcos-bump with: for REGION in us-east-1 us-west-1 us-west-2 do COUNT=3 if test us-west-1 = "${REGION}" then COUNT=2 fi for INDEX in 1 2 3 4 do NAME="do-not-delete-shared-vpc-${INDEX}" aws --region "${REGION}" cloudformation create-stack --stack-name "${NAME}" --template-body "$(cat upi/aws/cloudformation/01_vpc.yaml)" --parameters "ParameterKey=AvailabilityZoneCount,ParameterValue=${COUNT}" >/dev/null aws --region "${REGION}" cloudformation wait stack-create-complete --stack-name "${NAME}" SUBNETS="$(aws --region "${REGION}" cloudformation describe-stacks --stack-name "${NAME}" | jq -c '[.Stacks[].Outputs[] | select(.OutputKey | endswith("SubnetIds")).OutputValue | split(",")[]]' | sed "s/\"/'/g")" echo "${REGION}_$((INDEX - 1))) subnets=\"${SUBNETS}\";;" done done which spits out: us-east-2_0) subnets="['subnet-0faf6d16c378ee7a7','subnet-0e104572db1b7d092','subnet-014ca96c04f36adec','subnet-0ea06057dceadfe8e','subnet-0689efe5e1f9f4212','subnet-0d36bb8edbcb3d916']";; us-east-2_1) subnets="['subnet-085787cc4b80b84b2','subnet-09dfbf66e8f6e5b50','subnet-0db5d90ff3087444e','subnet-047f15f2a0210fbe0','subnet-0bf13f041c4233849','subnet-0e2a5320549e289d8']";; ... The COUNT bit for us-west-1 is because our CI account only has access to two zones there: $ aws --region us-west-1 ec2 describe-availability-zones --output text AVAILABILITYZONES us-west-1 available usw1-az3 us-west-1a AVAILABILITYZONES us-west-1 available usw1-az1 us-west-1b

ci-operator/templates/openshift/installer: Replacement shared subnets for new regions I'd created the previous subnets in 7e38260 (ci-operator/templates/openshift/installer: Shared subnets for new regions, 2020-01-23, openshift#6845), but forgot to add them to the reaper whitelist [1]. So the reaper just removed my old subnets. Recreating them here: $ export AWS_PROFILE=ci # or whatever you call it locally $ git fetch origin $ date --iso=m --utc 2020-01-30T17:09+0000 $ git checkout origin/release-4.3 $ git --no-pager log --oneline -1 2055609f9 (HEAD, origin/release-4.3) Merge pull request openshift#2928 from ashcrow/4.3-signed-rhcos-bump Clear out the old stacks: for REGION in us-east-2 us-west-1 us-west-2 do COUNT=3 if test us-west-1 = "${REGION}" then COUNT=2 fi for INDEX in 1 2 3 4 do NAME="do-not-delete-shared-vpc-${INDEX}" aws --region "${REGION}" cloudformation delete-stack --stack-name "${NAME}" aws --region "${REGION}" cloudformation wait stack-delete-complete --stack-name "${NAME}" done done I had to lean in manually and delete some instances in us-west-2's do-not-delete-shared-vpc-4 to unstick it. Then create the new subnets: for REGION in us-east-2 us-west-1 us-west-2 do COUNT=3 if test us-west-1 = "${REGION}" then COUNT=2 fi for INDEX in 1 2 3 4 do NAME="do-not-delete-shared-vpc-${INDEX}" aws --region "${REGION}" cloudformation create-stack --stack-name "${NAME}" --template-body "$(cat upi/aws/cloudformation/01_vpc.yaml)" --parameters "ParameterKey=AvailabilityZoneCount,ParameterValue=${COUNT}" >/dev/null aws --region "${REGION}" cloudformation wait stack-create-complete --stack-name "${NAME}" SUBNETS="$(aws --region "${REGION}" cloudformation describe-stacks --stack-name "${NAME}" | jq -c '[.Stacks[].Outputs[] | select(.OutputKey | endswith("SubnetIds")).OutputValue | split(",")[]]' | sed "s/\"/'/g")" echo "${REGION}_$((INDEX - 1))) subnets=\"${SUBNETS}\";;" done done 7e38260 had a us-east-1 typo in the commit message, fixed here. I actually used us-east-2 in that commit as well, and just fumbled the copy into the old commit message. Creation spit out: us-east-2_0) subnets="['subnet-0a568760cd74bf1d7','subnet-0320ee5b3bb78863e','subnet-015658a21d26e55b7','subnet-0c3ce64c4066f37c7','subnet-0d57b6b056e1ee8f6','subnet-0b118b86d1517483a']";; ... us-west-2_3) subnets="['subnet-072d00dcf02ad90a6','subnet-0ad913e4bd6ff53fa','subnet-09f90e069238e4105','subnet-064ecb1b01098ff35','subnet-068d9cdd93c0c66e6','subnet-0b7d1a5a6ae1d9adf']";; To generate the reaper whitelist [1], I used: for REGION in us-east-1 us-east-2 us-west-1 us-west-2 do for INDEX in 1 2 3 4 do NAME="do-not-delete-shared-vpc-${INDEX}" aws --region "${REGION}" resourcegroupstaggingapi get-resources --tag-filters "Key=aws:cloudformation:stack-name,Values=${NAME}" --query 'ResourceTagMappingList[].ResourceARN' | jq -r ".[] | split(\":\")[-1] | \" '\" + . + \"', # CI exclusion per DPP-4108, ${REGION} ${NAME}\"" done done | sort followed by some whitespace shuffling to get the comments aligned. [1]: openshift/li#3634 Private repository, sorry external folks.

openshift-ci-robot requested review from droslean and smarterclayton February 22, 2019 12:17

bbguimaraes force-pushed the mkpjpod branch from e32c268 to c2f5f84 Compare February 22, 2019 17:56

bbguimaraes force-pushed the mkpjpod branch from c2f5f84 to bef7891 Compare February 28, 2019 13:53

bbguimaraes force-pushed the mkpjpod branch from bef7891 to abdf96f Compare March 1, 2019 20:57

openshift-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Mar 1, 2019

hack/mkpjpod.sh: defaulting, general improvements

26f72f4

bbguimaraes force-pushed the mkpjpod branch from abdf96f to e2069a5 Compare March 5, 2019 14:11

openshift-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Mar 5, 2019

asciinema demos

e95bb62

bbguimaraes force-pushed the mkpjpod branch from e2069a5 to e95bb62 Compare March 5, 2019 14:12

bbguimaraes changed the title ~~[WIP] mkpjpod demos~~ mkpjpod demos Mar 5, 2019

openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 5, 2019

openshift-ci-robot assigned droslean Mar 5, 2019

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Mar 5, 2019

openshift-merge-robot merged commit 7465b12 into openshift:master Mar 5, 2019

bbguimaraes deleted the mkpjpod branch March 5, 2019 14:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mkpjpod demos #2928

mkpjpod demos #2928

bbguimaraes commented Feb 22, 2019

droslean commented Feb 22, 2019

bbguimaraes commented Feb 22, 2019

droslean commented Feb 25, 2019 •

edited

Loading

bbguimaraes commented Feb 25, 2019

bbguimaraes commented Feb 25, 2019

stevekuznetsov commented Feb 25, 2019

bbguimaraes commented Feb 25, 2019

bbguimaraes commented Feb 28, 2019

bbguimaraes commented Feb 28, 2019

droslean commented Feb 28, 2019

stevekuznetsov commented Feb 28, 2019

bbguimaraes commented Mar 1, 2019

droslean commented Mar 1, 2019

bbguimaraes commented Mar 1, 2019

droslean commented Mar 1, 2019

bbguimaraes commented Mar 4, 2019

bbguimaraes commented Mar 4, 2019

stevekuznetsov commented Mar 4, 2019

bbguimaraes commented Mar 5, 2019

droslean commented Mar 5, 2019

openshift-ci-robot commented Mar 5, 2019

mkpjpod demos #2928

mkpjpod demos #2928

Conversation

bbguimaraes commented Feb 22, 2019

droslean commented Feb 22, 2019

bbguimaraes commented Feb 22, 2019

droslean commented Feb 25, 2019 • edited Loading

bbguimaraes commented Feb 25, 2019

bbguimaraes commented Feb 25, 2019

stevekuznetsov commented Feb 25, 2019

bbguimaraes commented Feb 25, 2019

bbguimaraes commented Feb 28, 2019

bbguimaraes commented Feb 28, 2019

droslean commented Feb 28, 2019

stevekuznetsov commented Feb 28, 2019

bbguimaraes commented Mar 1, 2019

droslean commented Mar 1, 2019

bbguimaraes commented Mar 1, 2019

droslean commented Mar 1, 2019

bbguimaraes commented Mar 4, 2019

bbguimaraes commented Mar 4, 2019

stevekuznetsov commented Mar 4, 2019

bbguimaraes commented Mar 5, 2019

droslean commented Mar 5, 2019

openshift-ci-robot commented Mar 5, 2019

droslean commented Feb 25, 2019 •

edited

Loading