Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CI multicluster + OIDC environment #2619

Merged
merged 45 commits into from
Apr 5, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
721a679
Log wrong responses during the e2e tests
antgamdia Mar 29, 2021
b972339
Use pipeline params in CI. Add names to the steps
antgamdia Mar 29, 2021
66676ee
Add multicluster w/ oidc inconditionally
antgamdia Mar 29, 2021
1487874
Refactor login logic in e2e test using the envar
antgamdia Mar 29, 2021
7b6e377
Fix typo
antgamdia Mar 29, 2021
9a5ac98
Add missing import utils. Using imports
antgamdia Mar 29, 2021
83836d8
Use same prettier configuration as in the dashboard
antgamdia Mar 29, 2021
10e06b5
Use a more specific selector for the loading spinner
antgamdia Mar 29, 2021
e7049d6
Merge branch 'minor-dry-ci-fixes' into multicluster-ci
antgamdia Mar 29, 2021
788381d
Use require instead of es6 imports
antgamdia Mar 29, 2021
5f9c022
Use require instead of es6 imports
antgamdia Mar 29, 2021
30686dc
Remove unnecessary waits for the spinner to disappear
antgamdia Mar 29, 2021
03d3509
Merge branch 'master' into multicluster-ci
antgamdia Mar 29, 2021
25d933a
Remove unnecessary class
antgamdia Mar 29, 2021
2cbaa70
Add http redirect URI for CI purposes
antgamdia Mar 29, 2021
22664e4
Remove 'document' from utils lib, use 'page' instead
antgamdia Mar 29, 2021
80478e9
Fix wrong url
antgamdia Mar 29, 2021
f74443f
Ignore multicluster in gke
antgamdia Mar 29, 2021
37350dd
Increase timeout
antgamdia Mar 29, 2021
fe581bb
Fix typos
antgamdia Mar 30, 2021
9d67768
Better handling of envars
antgamdia Mar 30, 2021
7cf84c0
Add PR comments. Remove unused timeouts. Increase global timeout.
antgamdia Mar 30, 2021
830b574
Merge branch 'master' into multicluster-ci
antgamdia Mar 30, 2021
743af54
Create kubeapps-user rbac during the cluster creation. Use OLM versio…
antgamdia Mar 30, 2021
2dd82a2
Add flag to the kubeapps installation
antgamdia Mar 30, 2021
1db5a59
Temporary removal of the upgrade test
antgamdia Mar 30, 2021
23ea30c
Remove some flags. Edit timeout
antgamdia Mar 30, 2021
7383fae
Add temporary manual deletion of some resources
antgamdia Mar 30, 2021
026b29d
Add retry in k8s_wait_for_deployment
antgamdia Mar 30, 2021
c1656c2
Remove workaround
antgamdia Mar 30, 2021
79fa463
Add wait
antgamdia Mar 30, 2021
264d8a2
Add timeout to k8s_wait_for_deployment and remove sleep
antgamdia Mar 30, 2021
e6a1c1e
Increase wait until retry
antgamdia Mar 30, 2021
2775a31
Add watch to k8s_wait_for_deployment
antgamdia Mar 30, 2021
af61649
Add temporary workaround
antgamdia Mar 30, 2021
386f27a
Fix workaround
antgamdia Mar 30, 2021
0044b68
Add postgresqlPassword
antgamdia Mar 30, 2021
13bdad3
Add try/catch because of failing tests
antgamdia Mar 30, 2021
732bb33
Add test sequencer
antgamdia Mar 30, 2021
3320a08
Fix sequencer
antgamdia Mar 30, 2021
7086d6f
Remove unnecessary workaround
antgamdia Mar 30, 2021
3223042
Decrease sleep when retrying k8s_wait_for_deployment
antgamdia Apr 5, 2021
12322e7
Rename test cases to explicitize the order
antgamdia Apr 5, 2021
a63c0e4
Remove try/catch after the release 2.3
antgamdia Apr 5, 2021
bd1c0a1
Change button selectors
antgamdia Apr 5, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 91 additions & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -215,13 +215,19 @@ install_cluster: &install_cluster
run:
name: "Install cluster"
command: |
sed -i "s/172.18.0.2/$DEFAULT_DEX_IP/g" ./docs/user/manifests/kubeapps-local-dev-apiserver-config.yaml
{
echo "Creating cluster..."
kind create cluster --image kindest/node:${K8S_KIND_VERSION} --name kubeapps-ci --config=./docs/user/manifests/kubeapps-local-dev-apiserver-config.yaml --kubeconfig=${HOME}/.kube/kind-config-kubeapps-ci --retain --wait 120s &&
kubectl --context kind-kubeapps-ci --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci apply -f ./docs/user/manifests/kubeapps-local-dev-users-rbac.yaml &&

kubectl --context kind-kubeapps-ci --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci apply -f ./docs/user/manifests/ingress-nginx-kind-with-large-proxy-buffers.yaml &&
sleep 5 &&
kubectl --context kind-kubeapps-ci --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci wait --namespace ingress-nginx --for=condition=ready pod --selector=app.kubernetes.io/component=controller --timeout=120s &&

kubectl --context kind-kubeapps-ci --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci delete rolebinding kubeapps-user -n kubeapps-user-namespace &&
kubectl --context kind-kubeapps-ci --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci create rolebinding kubeapps-view-secret-oidc --role view-secrets --user oidc:kubeapps-user@example.com &&
kubectl --context kind-kubeapps-ci --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci create clusterrolebinding kubeapps-view-oidc --clusterrole=view --user oidc:kubeapps-user@example.com &&
echo "Cluster created"
} || {
echo "Cluster creation failed, retrying..."
Expand All @@ -231,6 +237,10 @@ install_cluster: &install_cluster
kubectl --context kind-kubeapps-ci --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci apply -f ./docs/user/manifests/ingress-nginx-kind-with-large-proxy-buffers.yaml &&
sleep 5 &&
kubectl wait --namespace ingress-nginx --for=condition=ready pod --selector=app.kubernetes.io/component=controller --timeout=120s &&

kubectl --context kind-kubeapps-ci --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci delete rolebinding kubeapps-user -n kubeapps-user-namespace &&
kubectl --context kind-kubeapps-ci --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci create rolebinding kubeapps-view-secret-oidc --role view-secrets --user oidc:kubeapps-user@example.com &&
kubectl --context kind-kubeapps-ci --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci create clusterrolebinding kubeapps-view-oidc --clusterrole=view --user oidc:kubeapps-user@example.com &&
echo "Cluster created"
} || {
echo "Error while creating the cluster after retry"
Expand All @@ -239,9 +249,62 @@ export_cluster_variables: &export_cluster_variables
run:
name: "Export cluster variables"
command: |
DEX_IP=`docker network inspect kind | jq '.[0].IPAM.Config[0].Gateway' | sed 's/"//g' | awk -F. '{ print $1"."$2"."$3"."$4+1 }'`
ADDITIONAL_CLUSTER_IP=`docker network inspect kind | jq '.[0].IPAM.Config[0].Gateway' | sed 's/"//g' | awk -F. '{ print $1"."$2"."$3"."$4+2 }'`

echo DEFAULT_DEX_IP=$DEFAULT_DEX_IP
echo DEX_IP=$DEX_IP
echo ADDITIONAL_CLUSTER_IP=$ADDITIONAL_CLUSTER_IP

# If running kubectl without args, use the default "kubeapps-ci" cluster
cp ${HOME}/.kube/kind-config-kubeapps-ci ${HOME}/.kube/config
kubectl config set-context kind-kubeapps-ci

# If the default IP the proper one, the multicluster setup will fail
if [ "$DEFAULT_DEX_IP" != "$DEX_IP" ]; then echo "Default IP does not match with current IP used in Kind"; exit 1; fi

echo "export DEFAULT_DEX_IP=${DEFAULT_DEX_IP}" >> $BASH_ENV
echo "export DEX_IP=${DEX_IP}" >> $BASH_ENV
echo "export ADDITIONAL_CLUSTER_IP=${ADDITIONAL_CLUSTER_IP}" >> $BASH_ENV
install_additional_cluster: &install_additional_cluster
run:
name: "Install additional cluster"
command: |
sed -i "s/172.18.0.2/$DEFAULT_DEX_IP/g" ./docs/user/manifests/kubeapps-local-dev-additional-apiserver-config.yaml
{
echo "Creating additional cluster..."
kind create cluster --image kindest/node:${K8S_KIND_VERSION} --name kubeapps-ci-additional --config=./docs/user/manifests/kubeapps-local-dev-additional-apiserver-config.yaml --kubeconfig=${HOME}/.kube/kind-config-kubeapps-ci-additional --retain --wait 120s &&
kubectl --context kind-kubeapps-ci-additional --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci-additional apply --kubeconfig=${HOME}/.kube/kind-config-kubeapps-ci-additional -f ./docs/user/manifests/kubeapps-local-dev-users-rbac.yaml &&
kubectl --context kind-kubeapps-ci-additional --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci-additional apply --kubeconfig=${HOME}/.kube/kind-config-kubeapps-ci-additional -f ./docs/user/manifests/kubeapps-local-dev-namespace-discovery-rbac.yaml &&

kubectl --context kind-kubeapps-ci-additional --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci-additional delete rolebinding kubeapps-user -n kubeapps-user-namespace &&
kubectl --context kind-kubeapps-ci-additional --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci-additional create rolebinding kubeapps-view-secret-oidc --role view-secrets --user oidc:kubeapps-user@example.com &&
kubectl --context kind-kubeapps-ci-additional --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci-additional create clusterrolebinding kubeapps-view-oidc --clusterrole=view --user oidc:kubeapps-user@example.com &&
echo "Additional cluster created"
} || {
echo "Additional cluster creation failed, retrying..."
kind delete clusters kubeapps-ci-additional || true
kind create cluster --image kindest/node:${K8S_KIND_VERSION} --name kubeapps-ci-additional --config=./docs/user/manifests/kubeapps-local-dev-additional-apiserver-config.yaml --kubeconfig=${HOME}/.kube/kind-config-kubeapps-ci-additional --retain --wait 120s &&
kubectl --context kind-kubeapps-ci-additional --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci-additional apply --kubeconfig=${HOME}/.kube/kind-config-kubeapps-ci-additional -f ./docs/user/manifests/kubeapps-local-dev-users-rbac.yaml &&
kubectl --context kind-kubeapps-ci-additional --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci-additional apply --kubeconfig=${HOME}/.kube/kind-config-kubeapps-ci-additional -f ./docs/user/manifests/kubeapps-local-dev-namespace-discovery-rbac.yaml &&

kubectl --context kind-kubeapps-ci-additional --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci-additional delete rolebinding kubeapps-user -n kubeapps-user-namespace &&
kubectl --context kind-kubeapps-ci-additional --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci-additional create rolebinding kubeapps-view-secret-oidc --role view-secrets --user oidc:kubeapps-user@example.com &&
kubectl --context kind-kubeapps-ci-additional --kubeconfig ${HOME}/.kube/kind-config-kubeapps-ci-additional create clusterrolebinding kubeapps-view-oidc --clusterrole=view --user oidc:kubeapps-user@example.com &&
echo "Additional cluster created"
} || {
echo "Error while creating the additional cluster after retry"
}

copy_apiserver_certificates: &copy_apiserver_certificates
run:
name: "Copy apiserver certificates"
command: |
# dex will be running on the same node as the API server in the dev environment, so we can reuse the key and cert from the apiserver
docker cp kubeapps-ci-control-plane:/etc/kubernetes/pki/apiserver.crt ./devel/dex.crt
docker cp kubeapps-ci-control-plane:/etc/kubernetes/pki/apiserver.key ./devel/dex.key
sudo chown circleci ./devel/dex.key
sudo chown circleci ./devel/dex.crt
install_kubectl: &install_kubectl
run:
name: "Install kubectl"
Expand All @@ -264,6 +327,25 @@ install_mkcert: &install_mkcert
chmod +x "mkcert-${MKCERT_VERSION}-linux-amd64"
sudo mv "mkcert-${MKCERT_VERSION}-linux-amd64" /usr/local/bin/mkcert
mkcert -install
install_multicluster_deps: &install_multicluster_deps
run:
name: "Install multicluster deps"
command: |
sed -i -e "s/172.18.0.2/$DEFAULT_DEX_IP/g;s/localhost/kubeapps-ci.kubeapps/g" ./docs/user/manifests/kubeapps-local-dev-dex-values.yaml
helm repo add stable https://charts.helm.sh/stable

# Install dex
kubectl create namespace dex
helm install dex stable/dex --namespace dex --values ./docs/user/manifests/kubeapps-local-dev-dex-values.yaml

# Install openldap
kubectl create namespace ldap
helm install ldap stable/openldap --namespace ldap

# Create certs
kubectl -n dex create secret tls dex-web-server-tls --key ./devel/dex.key --cert ./devel/dex.crt
mkcert -key-file ./devel/localhost-key.pem -cert-file ./devel/localhost-cert.pem localhost kubeapps-ci.kubeapps $DEFAULT_DEX_IP

run_e2e_tests: &run_e2e_tests
run:
name: "Run e2e tests script"
Expand All @@ -276,7 +358,7 @@ run_e2e_tests: &run_e2e_tests
IMG_MODIFIER=""
fi

if ./script/e2e-test.sh $OLM_VERSION $DEV_TAG $IMG_MODIFIER; then
if ./script/e2e-test.sh $USE_MULTICLUSTER_OIDC_ENV $OLM_VERSION $DEFAULT_DEX_IP $ADDITIONAL_CLUSTER_IP $DEV_TAG $IMG_MODIFIER; then
# Test success
echo "export TEST_RESULT=$?" >> $BASH_ENV
else
Expand Down Expand Up @@ -344,7 +426,9 @@ local_e2e_steps: &local_e2e_steps
- <<: *install_kind
- <<: *install_kubectl
- <<: *install_cluster
- <<: *copy_apiserver_certificates
# Create the "kubeapps-ci-additional" cluster
- <<: *install_additional_cluster
# Export variables and kubeconfig
- <<: *export_cluster_variables
- <<: *install_mkcert
Expand All @@ -355,6 +439,7 @@ local_e2e_steps: &local_e2e_steps
- run:
name: Load CI images in the cluster
command: for image in /tmp/images/*; do kind load image-archive "$image" --name kubeapps-ci; done
- <<: *install_multicluster_deps
- <<: *run_e2e_tests
- store_artifacts:
path: integration/reports
Expand Down Expand Up @@ -459,6 +544,7 @@ jobs:
DEFAULT_DEX_IP: "172.18.0.2"
KUBEAPPS_DB: "postgresql"
TEST_UPGRADE: "1"
USE_MULTICLUSTER_OIDC_ENV: "true"
<<: *common_envars
parameters:
number:
Expand All @@ -469,26 +555,30 @@ jobs:
environment:
GKE_BRANCH: "1.15"
KUBEAPPS_DB: "postgresql"
USE_MULTICLUSTER_OIDC_ENV: "false"
<<: *common_envars
GKE_1_15_LATEST_RELEASE:
<<: *gke_test
environment:
GKE_BRANCH: "1.15"
KUBEAPPS_DB: "postgresql"
TEST_LATEST_RELEASE: 1
USE_MULTICLUSTER_OIDC_ENV: "false"
<<: *common_envars
GKE_1_16_MASTER:
<<: *gke_test
environment:
GKE_BRANCH: "1.16"
KUBEAPPS_DB: "postgresql"
USE_MULTICLUSTER_OIDC_ENV: "false"
<<: *common_envars
GKE_1_16_LATEST_RELEASE:
<<: *gke_test
environment:
GKE_BRANCH: "1.16"
KUBEAPPS_DB: "postgresql"
TEST_LATEST_RELEASE: 1
USE_MULTICLUSTER_OIDC_ENV: "false"
<<: *common_envars
sync_chart:
docker:
Expand Down
3 changes: 3 additions & 0 deletions docs/user/manifests/kubeapps-local-dev-dex-values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ config:
# one.
- id: default
redirectURIs:
- 'http://localhost/oauth2/callback'
- 'https://localhost/oauth2/callback'
name: 'Default-Cluster'
secret: ZXhhbXBsZS1hcHAtc2VjcmV0
Expand All @@ -27,6 +28,7 @@ config:
# a trusted peer.
- id: second-cluster
redirectURIs:
- 'http://localhost/oauth2/callback'
- 'https://localhost/oauth2/callback'
name: 'Second-Cluster'
secret: ZXhhbXBsZS1hcHAtc2VjcmV0LXNlY29uZC1jbHVzdGVy
Expand All @@ -37,6 +39,7 @@ config:
# returned id_token.
- id: third-cluster
redirectURIs:
- 'http://localhost/oauth2/callback'
- 'https://localhost/oauth2/callback'
name: 'Third-Cluster'
secret: ZXhhbXBsZS1hcHAtc2VjcmV0LXNlY29uZC1jbHVzdGVy
Expand Down
5 changes: 5 additions & 0 deletions integration/.prettierrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
module.exports = {
printWidth: 100,
trailingComma: "all",
arrowParens: "avoid",
};
3 changes: 1 addition & 2 deletions integration/args.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,5 @@ module.exports = {
waitTimeout: process.env.INTEGRATION_WAIT_TIMEOUT || 60000,
headless: process.env.INTEGRATION_HEADLESS != "false",
retryAttempts: process.env.INTEGRATION_RETRY_ATTEMPTS || 0,
screenshotsFolder:
process.env.INTEGRATION_SCREENSHOTS_FOLDER || "reports/screenshots",
screenshotsFolder: process.env.INTEGRATION_SCREENSHOTS_FOLDER || "reports/screenshots",
};
2 changes: 1 addition & 1 deletion integration/jest-puppeteer.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ const { headless } = require("./args");
module.exports = {
launch: {
headless,
args: ["--no-sandbox", "--window-size=1200,780"],
args: ["--no-sandbox", "--window-size=1200,780", "--ignore-certificate-errors"],
},
browserContext: "incognito",
};
1 change: 1 addition & 0 deletions integration/jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,6 @@ module.exports = {
globalTeardown: "jest-environment-puppeteer/teardown",
testEnvironment: "./jest.environment.js",
testRunner: "jest-circus/runner",
testSequencer: "./jest.sequencer.js",
setupFilesAfterEnv: ["./jest.setup.js"],
};
29 changes: 10 additions & 19 deletions integration/jest.environment.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,7 @@ const waitOn = require("wait-on");
const PuppeteerEnvironment = require("jest-environment-puppeteer");
require("jest-circus");

const {
retryAttempts,
endpoint,
waitTimeout,
screenshotsFolder,
} = require("./args");
const { retryAttempts, endpoint, waitTimeout, screenshotsFolder } = require("./args");

// Create an environment to store a screenshot of the page if the current test
// failed.
Expand All @@ -31,7 +26,7 @@ class ScreenshotOnFailureEnvironment extends PuppeteerEnvironment {
// Check the server is up before running the test suite
console.log(
`Waiting ${endpoint} to be ready before running the tests
(${waitTimeout / 1000}s)`
(${waitTimeout / 1000}s)`,
);
await waitOn({
resources: [endpoint],
Expand All @@ -54,32 +49,30 @@ class ScreenshotOnFailureEnvironment extends PuppeteerEnvironment {
height: 780,
deviceScaleFactor: 1,
});
await this.global.page.setDefaultTimeout(8000);
await this.global.page.setDefaultTimeout(30000); // 30s is the default value
this.global.page
.on("console", async (message) => {
.on("console", async message => {
if (message.type() === "error") {
console.error(`${message.type().toUpperCase()} ${message.text()}`);
// Code from https://github.com/puppeteer/puppeteer/issues/3397#issuecomment-429325514
const args = await message.args();
args.forEach(async (arg) => {
args.forEach(async arg => {
const val = await arg.jsonValue();
// value is serializable
if (JSON.stringify(val) !== JSON.stringify({})) console.log(val);
// value is unserializable (or an empty oject)
else {
const { type, subtype, description } = arg._remoteObject;
console.log(
`type: ${type}, subtype: ${subtype}, description:\n ${description}`
);
console.log(`type: ${type}, subtype: ${subtype}, description:\n ${description}`);
}
});
}
})
.on("pageerror", ({ message }) => console.log(message))
.on("requestfailed", (request) =>
console.log(`${request.failure().errorText} ${request.url()}`)
.on("requestfailed", request =>
console.log(`${request.failure().errorText} ${request.url()}`),
)
.on("response", (response) => {
.on("response", response => {
if (response.status() >= 400) {
console.log(`${response.status()} in ${response.url()}`);
}
Expand All @@ -96,9 +89,7 @@ class ScreenshotOnFailureEnvironment extends PuppeteerEnvironment {
async handleTestEvent(event, state) {
if (event.name == "test_fn_failure") {
if (state.currentlyRunningTest.invocations > retryAttempts) {
const testName = state.currentlyRunningTest.name
.toLowerCase()
.replace(/ /g, "-");
const testName = state.currentlyRunningTest.name.toLowerCase().replace(/ /g, "-");
// Take a screenshot at the point of failure
await this.global.page.screenshot({
path: path.join(__dirname, `${screenshotsFolder}/${testName}.png`),
Expand Down
10 changes: 10 additions & 0 deletions integration/jest.sequencer.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
const Sequencer = require("@jest/test-sequencer").default;

class CustomSequencer extends Sequencer {
sort(tests) {
const copyTests = Array.from(tests);
return copyTests.sort((testA, testB) => (testA.path > testB.path ? 1 : -1));
Comment on lines +3 to +6
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Despite the tests use to be executed in alphabetical order, there is no actual guarantee of that. It was causing a failure in the multicluster test because the secret created in the secret-repo test wasn't being found.
This sequencer just ensures this alphabetical order (I've pushed the integration img with this dep as well)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are saying that creating a private repository causes an error in the multicluster scenario? (That sounds like a bug we should fix). If that's the case, can you open an issue for it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't yet support app repos (let alone secret ones) on other clusters (#1982) because the interface to sync with the asset svc is via the DB (#2394) (left over from the read-only monocular).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification, I always forget about this issue.
Perhaps we should name the tests like "01-xxxx.js", "00-xxx.js" just to make clear that they are being executed in order.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know that private repositories are not supported in the second cluster but what I don't understand is why the order of the tests affects the result. I thought that creating a private repo in the default cluster caused an error when trying to deploy in the second cluster but that may not be it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, ok, I see your point now and it may be an issue as you said:

When creating a private repo first, we are creating the docker credentials. Somehow, when deploying in the second cluster, it tries to retrieve these credentials (not available, though), and fails, therefore.

image

Let me replicate the issue in my dev environment and will file an issue with the details.

However, with regards to this PR, I think (for the sake of replicability) to still define an explicit order during the test suite execution.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't able to reproduce this issue in my dev environment. I set up the multicluster env and: 1) tried to manually create the private repo and then a new deployment; 2) run the e2e test directly from yarn start ....
In both cases, the credentials are being properly stored and the app deploys seamlessly as well.

That's odd... :S

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably added a namespaced private repository. If you create it in the kubeapps namespace, you are likely able to reproduce the issue.

}
}

module.exports = CustomSequencer;
5 changes: 3 additions & 2 deletions integration/jest.setup.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
const { endpoint } = require("./args");
const { setDefaultOptions } = require("expect-puppeteer");

setDefaultOptions({ timeout: 4000 });
// Change timeout for Puppeteer page.waitForXXX functions from 0.5s to 30s
setDefaultOptions({ timeout: 30000 });

// endpoint argument is mandatory
if (endpoint == null || endpoint == "") {
Expand All @@ -13,7 +14,7 @@ if (endpoint == null || endpoint == "") {
global.endpoint = endpoint;

// Helper to get the proper endpoint
global.getUrl = (path) => `${global.endpoint}${path}`;
global.getUrl = path => `${global.endpoint}${path}`;

// Timeout for a test
jest.setTimeout(120000);
7 changes: 6 additions & 1 deletion integration/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,20 @@
"private": true,
"scripts": {
"start": "jest --runInBand --ci --colors",
"start:window": "INTEGRATION_HEADLESS=false jest --runInBand"
"start:window": "INTEGRATION_HEADLESS=false jest --runInBand",
"prettier": "prettier --write './**/*.{js, json}'"
},
"dependencies": {
"@jest/test-sequencer": "^26.6.3",
"axios": "^0.21.1",
"expect-puppeteer": "^4.4.0",
"jest": "^26.6.3",
"jest-circus": "^26.6.3",
"jest-puppeteer": "^4.4.0",
"puppeteer": "^8.0.0",
"wait-on": "^5.3.0"
},
"devDependencies": {
"prettier": "^2.2.1"
}
}
30 changes: 30 additions & 0 deletions integration/use-cases/01-add-multicluster-deployment.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
const utils = require("./lib/utils");

test("Deploys an application with the values by default", async () => {
await utils.login(
page,
process.env.USE_MULTICLUSTER_OIDC_ENV,
"/",
"",
"kubeapps-operator@example.com",
"password",
);

// Change cluster using ui
await expect(page).toClick(".kubeapps-nav-link");

await page.select('select[name="clusters"]', "second-cluster");

await expect(page).toClick("cds-button", { text: "Change Context" });

await expect(page).toClick("a", { text: "Catalog" });

await expect(page).toMatchElement("a", { text: "apache", timeout: 60000 });
await expect(page).toClick("a", { text: "apache" });

await expect(page).toClick("cds-button", { text: "Deploy" });

await expect(page).toClick("cds-button", { text: "Deploy" });

await expect(page).toMatch("Ready", { timeout: 60000 });
});
Loading