feat(cloudquery-usage): Track CloudQuery plugin usage #1394
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
akash1810
force-pushed
the
aa/cloudquery-usage
branch
2 times, most recently
from
bd1757b to
444a02e
Compare
akash1810
force-pushed
the
aa/cloudquery-usage
branch
from
444a02e to
49af7cb
Compare
akash1810
commented
Jan 25, 2025
| @@ -57,3 +57,5 @@ ENABLE_MESSAGING=false | |||
| INTERACTIVES_COUNT=3 | |||
|
|
|||
| STACK=deploy | |||
|
|
|||
| CQ_API_KEY_PATH=/CODE/deploy/service-catalogue/cloudquery-api-key | |||
There was a problem hiding this comment.
Locally, use the API key for CODE.
| @@ -1,3 +1,3 @@ | |||
| # Please do not edit this file manually | |||
| # It should be added in your version-control system (i.e. Git) | |||
| # It should be added in your version-control system (e.g., Git) | |||
There was a problem hiding this comment.
This file is generated by Prisma. This must be the first time creating a migration since #1389.
Creates a new service that writes to a new database table `cloudquery_plugin_usage` tracking the number of paid rows are synced per plugin, per day.
akash1810
force-pushed
the
aa/cloudquery-usage
branch
from
49af7cb to
bed5ea1
Compare
jorgeazevedo
approved these changes
Jan 27, 2025
I've just ran a backfill of PROD from 01 Jan 2024 to now (26 Jan 2025). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this change and why?
Creates a new AWS Lambda that tracks plugin usage per day by calling the GetTeamUsageSummary endpoint and storing the results in a new database table
cloudquery_plugin_usage.By default, yesterday's data is collected. This can be overridden by setting the
START_DATEandEND_DATEenvironment variables, enabling us to backfill data.The contents of this table will look something like this:
This allows us to monitor our daily usage to ensure we do not breach our contract.
Unfortunately, the API does not return the version of the plugin used. This would be helpful as, when looking through the data, we can see a big spike from 04 July 2024, which can be attributed to #1153.
How has it been verified?
I've deployed to CODE and ran the lambda to backfill data from 01 Jan 2024 to now.