Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable Druid to use mock GCS server to run e2e tests #741

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Enable Druid to use mock GCS server to run e2e tests #741

wants to merge 5 commits into from

Conversation

anveshreddy18
Copy link
Contributor

@anveshreddy18 anveshreddy18 commented Jan 1, 2024 •
edited
Loading

How to categorize this PR?

/area dev-productivity
/area testing
/kind test

What this PR does / why we need it:

This PR enables druid to run e2e tests for GCS locally with the help of a GCS Emulator called fake-gcs-server.

To run e2e tests with GCS Emulator as object store, do make ci-e2e-kind-gcs.

Which issue(s) this PR fixes:
Fixes #738

Special notes for your reviewer:

Release note:

Enable Druid to use mock GCS server to run e2e tests using "make ci-e2e-kind-gcs"

@anveshreddy18 anveshreddy18 requested a review from a team as a code owner January 1, 2024 11:58
@gardener-robot gardener-robot added needs/review Needs review area/dev-productivity Developer productivity related (how to improve development) area/testing Testing related kind/test Test labels Jan 1, 2024
@anveshreddy18 anveshreddy18 self-assigned this Jan 1, 2024
@gardener-robot gardener-robot added the size/m Size of pull request is medium (see gardener-robot robot/bots/size.py) label Jan 1, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Jan 1, 2024
renormalize
renormalize requested changes Jan 2, 2024
View reviewed changes
hack/ci-gcs-e2e-kind.sh Outdated Show resolved Hide resolved
@gardener-robot gardener-robot added the needs/changes Needs (more) changes label Jan 2, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jan 2, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jan 2, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Jan 3, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Jan 3, 2024
@gardener-robot gardener-robot added size/l Size of pull request is large (see gardener-robot robot/bots/size.py) needs/second-opinion Needs second review by someone else and removed size/m Size of pull request is medium (see gardener-robot robot/bots/size.py) labels Jan 4, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jan 4, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jan 4, 2024
@ishan16696 ishan16696 added the reviewed/do-not-merge Has no approval for merging as it may break things, be of poor quality or have (ext.) dependencies label Jan 4, 2024
@anveshreddy18 anveshreddy18 added the kind/enhancement Enhancement, improvement, extension label Jan 18, 2024
@gardener-robot-ci-3 gardener-robot-ci-3 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Feb 13, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Mar 15, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Mar 15, 2024
@gardener-robot-ci-3 gardener-robot-ci-3 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 16, 2024
@anveshreddy18
Copy link
Contributor Author

Thanks for the review @seshachalam-yv, can you pls test now. Earlier there was a hashtag infront of the svcacc.json field in the secret, hence you saw the failure to read the creds. I've now removed that, and updated the docs. Pls follow the latest doc to run it successfully.

@seshachalam-yv
Copy link
Contributor

Could you create a new PR to bump the version of etcd-backup-restore?

@anveshreddy18 anveshreddy18 mentioned this pull request Jul 16, 2024
Merged
renormalize
renormalize requested changes Jul 17, 2024
View reviewed changes
test/e2e/utils.go Outdated Show resolved Hide resolved
internal/common/constants.go Outdated Show resolved Hide resolved
@gardener-robot-ci-3 gardener-robot-ci-3 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 17, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 17, 2024
renormalize
renormalize requested changes Jul 18, 2024
View reviewed changes
Copy link
Member

@renormalize renormalize left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's conflicts with the latest commit in master again now, can you resolve the conflicts? Thanks.

docs/development/local-e2e-tests.md Outdated Show resolved Hide resolved
@anveshreddy18
Enable Druid to use mock GCS server to run e2e tests
d984850 
Address review comments

Create gcp secret to use emulator for druid local setup

Add documentation for usage of fake-gcs-server for e2e tests

Add documentation to setup etcd-druid locally using fake-gcs-server

Propagate changes made to etcdbr PR#697

Rename EMULATOR_ENABLED to GOOGLE_EMULATOR_ENABLED

Upgrade etcdbr dependency to 0.29.0 for emulator support in e2e tests

Address review comments by @seshachalam-yv

Revert the etcdbr bump & address review comments
@anveshreddy18
Address review comments
3af95c4
@gardener-robot gardener-robot added size/l Size of pull request is large (see gardener-robot robot/bots/size.py) and removed size/m Size of pull request is medium (see gardener-robot robot/bots/size.py) labels Jul 19, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Jul 19, 2024
seshachalam-yv
seshachalam-yv requested changes Jul 19, 2024
View reviewed changes
go.mod Outdated Show resolved Hide resolved
hack/e2e-test/infrastructure/kind/cluster.yaml Outdated Show resolved Hide resolved
test/e2e/utils.go Outdated Show resolved Hide resolved
@gardener-robot-ci-3 gardener-robot-ci-3 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 22, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 22, 2024
@anveshreddy18
Copy link
Contributor Author

/retest

@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 25, 2024
@gardener-robot-ci-3 gardener-robot-ci-3 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 25, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 25, 2024
@gardener-robot-ci-3 gardener-robot-ci-3 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 25, 2024
@anveshreddy18
Copy link
Contributor Author

/hold

Needs some changes as etcdbr will be modified to depend on mounted secret instead of env vars from druid for endpoint & emulatorEnabled vars. That needs to be accommodated here as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@renormalize renormalize renormalize requested changes

@seshachalam-yv seshachalam-yv seshachalam-yv requested changes

Requested changes must be addressed to merge this pull request.

Assignees

@anveshreddy18 anveshreddy18

@seshachalam-yv seshachalam-yv

Labels
area/dev-productivity Developer productivity related (how to improve development) area/testing Testing related kind/enhancement Enhancement, improvement, extension kind/test Test needs/changes Needs (more) changes needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) needs/rebase Needs git rebase needs/review Needs review needs/second-opinion Needs second review by someone else reviewed/do-not-merge Has no approval for merging as it may break things, be of poor quality or have (ext.) dependencies size/l Size of pull request is large (see gardener-robot robot/bots/size.py)
Projects
None yet
Milestone
v0.24.0
Development

Successfully merging this pull request may close these issues.

[Feature] Enable Druid to use mock GCS server to run e2e tests
9 participants
@anveshreddy18 @gardener-robot @seshachalam-yv @renormalize @gardener-robot-ci-1 @shreyas-s-rao @ishan16696 @gardener-robot-ci-2 @gardener-robot-ci-3