Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pubsub test #2696

Merged
merged 6 commits into from
Mar 12, 2022
Merged

Pubsub test #2696

merged 6 commits into from
Mar 12, 2022

Conversation

RamCohen
Copy link
Contributor

@RamCohen RamCohen commented Mar 1, 2022

chore: Add pubsub scaler e2e test

Fixes #1528

@RamCohen RamCohen requested a review from a team as a code owner March 1, 2022 15:27
@JorTurFer
Copy link
Member

JorTurFer commented Mar 1, 2022 •
edited by github-actions bot
Loading

/run-e2e gcp-pubsub.test.ts
Update: You can check the progres here

JorTurFer
JorTurFer requested changes Mar 1, 2022
View reviewed changes
Copy link
Member

@JorTurFer JorTurFer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a ton for this nice improvement!!!!!
Only small nits

tests/scalers/gcp-pubsub.test.ts Outdated Show resolved Hide resolved
tests/scalers/gcp-pubsub.test.ts Outdated Show resolved Hide resolved
@JorTurFer
Copy link
Member

JorTurFer commented Mar 1, 2022 •
edited by github-actions bot
Loading

/run-e2e gcp-pubsub.test.ts
Update: You can check the progres here

JorTurFer
JorTurFer approved these changes Mar 2, 2022
View reviewed changes
Copy link
Member

@JorTurFer JorTurFer left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
♥️♥️♥️♥️

@JorTurFer
Copy link
Member

JorTurFer commented Mar 2, 2022 •
edited by github-actions bot
Loading

/run-e2e gcp-pubsub.test.ts

Update: You can check the progres here

zroubalik
zroubalik approved these changes Mar 2, 2022
View reviewed changes
Copy link
Member

@zroubalik zroubalik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks a lot for this @RamCohen !

@JorTurFer
Copy link
Member

JorTurFer commented Mar 2, 2022 •
edited by github-actions bot
Loading

/run-e2e gcp-pubsub.test.ts
Update: You can check the progres here

@JorTurFer
Copy link
Member

Tests look a bit unstables.
Could you take a look please @RamCohen ?

@zroubalik
Copy link
Member 

   ✔ creating the gcp-sdk pod should work.. (598ms)
error: unable to upgrade connection: container not found ("gcp-sdk-container")
error: unable to upgrade connection: container not found ("gcp-sdk-container")
error: unable to upgrade connection: container not found ("gcp-sdk-container")
ERROR: (gcloud.pubsub.subscriptions.create) You do not currently have an active account selected.
Please run:

  $ gcloud auth login

to obtain new credentials.

If you have already logged in with a different account:

    $ gcloud config set account ACCOUNT

to select an already authenticated account to use.

@JorTurFer
Copy link
Member

JorTurFer commented Mar 3, 2022 •
edited by github-actions bot
Loading

/run-e2e gcp-pubsub.test.ts
Update: You can check the progres here

RamCohen added 4 commits March 3, 2022 10:02
@RamCohen
chore: Add pubsub scaler e2e test
6849bc3 
Fixes kedacore#1528

Signed-off-by: Ram Cohen <ram.cohen@gmail.com>
@RamCohen
Keep the pubsub subscriber pod alive
167e0ed 
Signed-off-by: Ram Cohen <ram.cohen@gmail.com>
@RamCohen
Make e2e test work with the test GCP project
efdc357 
Signed-off-by: Ram Cohen <ram.cohen@gmail.com>
@RamCohen
Use methods from helpers utilities
b9bc175 
Signed-off-by: Ram Cohen <ram.cohen@gmail.com>
@zroubalik
Copy link
Member

zroubalik commented Mar 3, 2022 •
edited by github-actions bot
Loading

/run-e2e gcp-pubsub.test.ts
Update: You can check the progres here

@RamCohen RamCohen force-pushed the pubsub-test branch 2 times, most recently from 3891d1f to 0c79cdf Compare March 3, 2022 11:37
@RamCohen
Updated changelog
e994684 
Signed-off-by: Ram Cohen <ram.cohen@gmail.com>

Fix GCP authentication in test

Signed-off-by: Ram Cohen <ram.cohen@gmail.com>

Testing: Wait for GCP sdk pod to be ready

Signed-off-by: Ram Cohen <ram.cohen@gmail.com>

Remove key

Signed-off-by: Ram Cohen <ram.cohen@gmail.com>
@RamCohen
Small text fix
c2550b5 
Signed-off-by: Ram Cohen <ram.cohen@gmail.com>
@zroubalik
Copy link
Member

/run-e2e gcp-pubsub.test.ts Update: You can check the progres here

@RamCohen FYI

  ✔ creating the gcp-sdk pod should work.. (***.3s)
Activated service account credentials for: [keda-***@azure-deprecation-production.iam.gserviceaccount.com]
API [pubsub.googleapis.com] not enabled on project [538407947***74]. Would you 
like to enable and retry (this will take a few minutes)? (y/N)?  

ERROR: (gcloud.pubsub.topics.create) PERMISSION_DENIED: Cloud Pub/Sub API has not been used in project 538407947***74 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/pubsub.googleapis.com/overview?project=538407947***74 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
- '@type': type.googleapis.com/google.rpc.Help
  links:
  - description: Google developers console API activation
    url: https://console.developers.google.com/apis/api/pubsub.googleapis.com/overview?project=538407947***74
- '@type': type.googleapis.com/google.rpc.ErrorInfo
  domain: googleapis.com
  metadata:
    consumer: projects/538407947***74
    service: pubsub.googleapis.com
  reason: SERVICE_DISABLED
command terminated with exit code ***
API [pubsub.googleapis.com] not enabled on project [538407947***74]. Would you 
like to enable and retry (this will take a few minutes)? (y/N)?

@RamCohen
Copy link
Contributor Author

RamCohen commented Mar 3, 2022

Well, at least we are getting somewhere...
I'll push a revision which does the enablement

@RamCohen
Copy link
Contributor Author

RamCohen commented Mar 3, 2022

It's better to do it in the console.
Please enable Pubsub, GCS and stackdriver APIs

@zroubalik
Copy link
Member

It's better to do it in the console. Please enable Pubsub, GCS and stackdriver APIs

@tomkerkhove could you PTAL on this?

@JorTurFer
Copy link
Member

JorTurFer commented Mar 11, 2022 •
edited by github-actions bot
Loading

/run-e2e gcp-pubsub.test.ts
Update: You can check the progres here

@RamCohen
Copy link
Contributor Author

Current error:

ERROR: (gcloud.pubsub.topics.create) PERMISSION_DENIED: Cloud Pub/Sub API has not been used in project 538407947***74 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/pubsub.googleapis.com/overview?project=538407947***74 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

@JorTurFer
Copy link
Member

JorTurFer commented Mar 11, 2022 •
edited by github-actions bot
Loading

/run-e2e gcp-pubsub.test.ts
Update: You can check the progres here

@JorTurFer
Copy link
Member

🤦 ...
I enabled Pub/Sub lite instead of this...
Thanks for the link, now it's enabled. Retrying

@JorTurFer
Copy link
Member

JorTurFer commented Mar 11, 2022 •
edited
Loading

Also enabled these (picture is in Spanish but I think is understandable):
image

Do you think there is anyone missing @RamCohen ?

@RamCohen
Copy link
Contributor Author

don't think we need cloud reporting or profiling or pubsub lite
remember to also enable cloud storag

@JorTurFer
Copy link
Member

Is this?
image

@JorTurFer
Copy link
Member

JorTurFer commented Mar 11, 2022 •
edited by github-actions bot
Loading

/run-e2e gcp-pubsub.test.ts
Update: You can check the progres here

@JorTurFer
Copy link
Member

right now I keep enabled the API from the last picture, the link that you sent me about PubSub and stackdriver + stackdriver monitoring

@RamCohen
Copy link
Contributor Author

Still getting the same error. Are you sure you are enabling it for the right project (if there is more than one) ?

ERROR: (gcloud.pubsub.topics.create) PERMISSION_DENIED: Cloud Pub/Sub API has not been used in project 538407947***74 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/pubsub.googleapis.com/overview?project=538407947***74 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

@JorTurFer
Copy link
Member

JorTurFer commented Mar 11, 2022 •
edited
Loading

image
I think so.
Maybe credentials are not related with my account? Let me update the secret with another new google application credentials.
I'll try it locally before, so maybe tomorrow I will do it

@RamCohen
Copy link
Contributor Author

Go the the Pub/Sub service in the console like this:

image

@JorTurFer
Copy link
Member

JorTurFer commented Mar 11, 2022 •
edited by github-actions bot
Loading

/run-e2e gcp-pubsub.test.ts

Update: You can check the progres here

@JorTurFer
Copy link
Member

image
Let's wait this execution, I have updated the service account with another. I'm not totally sure if the previous was correct

@RamCohen
Copy link
Contributor Author

Seems Ok now :-)

@JorTurFer
Copy link
Member

JorTurFer commented Mar 11, 2022 •
edited by github-actions bot
Loading

/run-e2e gcp-pubsub.test.ts
Update: You can check the progres here

@JorTurFer
Copy link
Member

Yes, I think other the credentials were wrong because the email was different.

@JorTurFer
Copy link
Member

Thanks a ton for this e2e test!!! ❤️

@JorTurFer JorTurFer merged commit bc0b7f6 into kedacore:main Mar 12, 2022
@RamCohen RamCohen deleted the pubsub-test branch March 13, 2022 06:09
RamCohen added a commit to RamCohen/keda that referenced this pull request Mar 23, 2022
@RamCohen
Pubsub test (kedacore#2696)
d18862a 
Signed-off-by: Ram Cohen <ram.cohen@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JorTurFer JorTurFer JorTurFer approved these changes

@zroubalik zroubalik zroubalik approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add e2e test for Google Cloud Pub/Sub Scaler
3 participants
@RamCohen @JorTurFer @zroubalik