Send Event metrics via Telemetry#3979
Merged
hawflau merged 4 commits intoaws:customer-journeyfrom Jun 22, 2022
Merged
Conversation
hawflau
suggested changes
Jun 17, 2022
qingchm
reviewed
Jun 20, 2022
hawflau
approved these changes
Jun 21, 2022
Contributor
hawflau
left a comment
There was a problem hiding this comment.
LGTM! Thanks for making the changes!
qingchm
approved these changes
Jun 22, 2022
hawflau
added a commit
that referenced
this pull request
Aug 23, 2022
* Add project metadata tracking with telemetry (#3929) * Implement project metadata metrics * Implement metric metadata integration tests * Tweak methods so tests don't fail on Linux machines * Repair failing integration test * Implement proper exception handling for subprocesses * Format according to standard * Refactor function names and encryption logic * Remove unnecessary file * Add type hints to new functions * Fix retrieval process for directory name (#3962) * Implement event data structure and relevant unit tests (#3955) * Implement event data structure and relevant unit tests * Implement requested changes * Remove unused (useless) file * Remove unnecessary ABC class * Implement tracker methods (#3970) * Implement tracker methods * Improve readability of Event's __repr__ method * Add method to convert Event to dictionary for JSON-compatibility * Send Event metrics via Telemetry (#3979) * Send Event metrics via Telemetry * Add unit tests for event telemetry * Simplify event telemetry unit tests * Clean up unit tests * Add BuildRuntime event and trackers (#3994) * Add BuildRuntime Events * Remove unnecessary file * Prevent tracked Event 'spillage' from previous tests * Fix builder logic * Redefine BuildRuntime to focus on function runtimes * Remove unnecessary file (again) * Add UsedFeature event and trackers (#4003) * Add UsedFeature command and trackers * Remove LocalTest events Events regarding sam local will be implemented at a later time. * Prevent methods from failing if track_event fails * Reformat code to match current standard * Use correct string formatting * Use proper LOG message * Add thread lock to EventTracker (#4019) * Add thread lock to EventTracker * Place lock in appropriate methods * Refactor Event data structure (#4027) * Add new attributes to Event datastructure * Fix non-JSON-serializable field in Event * Fix repr to ensure SQL casting will work * Add comment for timestamp method clarity * Update tests to verify EventTracker thread lock is working (#4035) * Add send_events method to EventTracker (#4039) * Add framework for Event-sending method * Perform some general refactors and cleanup * Implement EventTracker Telemetry unit tests * Cleanup tests and unused imports * Update Metric name for consistency * Repair failing unit test * (Hopefully) repair broken integration test * *Actually* repair broken integration test I had the >= logic backwards lol * Fix integration test For sure this time * Really fix the integration test Windows development is not fun * Properly fix integration test * Reformat files * Implement requested changes * Add Lock logic to send_events method * Implement requested changes * Repair failing tests * Upgrade encryption algorithm from SHA-1 to SHA-256 (#4061) * Upgrade encryption algorithm from SHA1 to SHA256 * Make return value of hash JSON-serializable * Add long event tracker (#4067) * Send events via thread when at capacity (#4072) * Switch send_events to thread when at capacity * Repair failing unit test * Add `sam sync` Events (#4076) * Add sam sync Events * Refactor methods for tracking sync events * Improve SyncFlow tracking * Add additional SyncFlow value * Add workflow events (#4096) * Add workflow_config events * Refactor workflow Events * Add tests for verifying workflow events * Remove redundant (UsedFeature, ESBuild) event * Repair failing unit test * Rename workflow event to prevent ambiguity * Fix send_events logic (#4104) * Make minor tweaks before launch (#4111) * Apply minor changes (some major) * Add session_id for multithreaded send_events * Adjust tests for new multithreading * Fix failing integration test * *Actually* fix failing integration test * Properly fix integration test * This time for sure * Really really repair that test * OK this one SHOULD work I swear * This time for sure * FIX INTEGRATION TEST * IT'S A LIST * Remove unused imports * Add debug log for EventTracker session ID * Place debug log in better position Co-authored-by: Wing Fung Lau <4760060+hawflau@users.noreply.github.com>
torresxb1
pushed a commit
to torresxb1/aws-sam-cli
that referenced
this pull request
Sep 2, 2022
* Add project metadata tracking with telemetry (aws#3929) * Implement project metadata metrics * Implement metric metadata integration tests * Tweak methods so tests don't fail on Linux machines * Repair failing integration test * Implement proper exception handling for subprocesses * Format according to standard * Refactor function names and encryption logic * Remove unnecessary file * Add type hints to new functions * Fix retrieval process for directory name (aws#3962) * Implement event data structure and relevant unit tests (aws#3955) * Implement event data structure and relevant unit tests * Implement requested changes * Remove unused (useless) file * Remove unnecessary ABC class * Implement tracker methods (aws#3970) * Implement tracker methods * Improve readability of Event's __repr__ method * Add method to convert Event to dictionary for JSON-compatibility * Send Event metrics via Telemetry (aws#3979) * Send Event metrics via Telemetry * Add unit tests for event telemetry * Simplify event telemetry unit tests * Clean up unit tests * Add BuildRuntime event and trackers (aws#3994) * Add BuildRuntime Events * Remove unnecessary file * Prevent tracked Event 'spillage' from previous tests * Fix builder logic * Redefine BuildRuntime to focus on function runtimes * Remove unnecessary file (again) * Add UsedFeature event and trackers (aws#4003) * Add UsedFeature command and trackers * Remove LocalTest events Events regarding sam local will be implemented at a later time. * Prevent methods from failing if track_event fails * Reformat code to match current standard * Use correct string formatting * Use proper LOG message * Add thread lock to EventTracker (aws#4019) * Add thread lock to EventTracker * Place lock in appropriate methods * Refactor Event data structure (aws#4027) * Add new attributes to Event datastructure * Fix non-JSON-serializable field in Event * Fix repr to ensure SQL casting will work * Add comment for timestamp method clarity * Update tests to verify EventTracker thread lock is working (aws#4035) * Add send_events method to EventTracker (aws#4039) * Add framework for Event-sending method * Perform some general refactors and cleanup * Implement EventTracker Telemetry unit tests * Cleanup tests and unused imports * Update Metric name for consistency * Repair failing unit test * (Hopefully) repair broken integration test * *Actually* repair broken integration test I had the >= logic backwards lol * Fix integration test For sure this time * Really fix the integration test Windows development is not fun * Properly fix integration test * Reformat files * Implement requested changes * Add Lock logic to send_events method * Implement requested changes * Repair failing tests * Upgrade encryption algorithm from SHA-1 to SHA-256 (aws#4061) * Upgrade encryption algorithm from SHA1 to SHA256 * Make return value of hash JSON-serializable * Add long event tracker (aws#4067) * Send events via thread when at capacity (aws#4072) * Switch send_events to thread when at capacity * Repair failing unit test * Add `sam sync` Events (aws#4076) * Add sam sync Events * Refactor methods for tracking sync events * Improve SyncFlow tracking * Add additional SyncFlow value * Add workflow events (aws#4096) * Add workflow_config events * Refactor workflow Events * Add tests for verifying workflow events * Remove redundant (UsedFeature, ESBuild) event * Repair failing unit test * Rename workflow event to prevent ambiguity * Fix send_events logic (aws#4104) * Make minor tweaks before launch (aws#4111) * Apply minor changes (some major) * Add session_id for multithreaded send_events * Adjust tests for new multithreading * Fix failing integration test * *Actually* fix failing integration test * Properly fix integration test * This time for sure * Really really repair that test * OK this one SHOULD work I swear * This time for sure * FIX INTEGRATION TEST * IT'S A LIST * Remove unused imports * Add debug log for EventTracker session ID * Place debug log in better position Co-authored-by: Wing Fung Lau <4760060+hawflau@users.noreply.github.com>
hawflau
added a commit
that referenced
this pull request
May 17, 2023
* Add project metadata tracking with telemetry (#3929) * Implement project metadata metrics * Implement metric metadata integration tests * Tweak methods so tests don't fail on Linux machines * Repair failing integration test * Implement proper exception handling for subprocesses * Format according to standard * Refactor function names and encryption logic * Remove unnecessary file * Add type hints to new functions * Fix retrieval process for directory name (#3962) * Implement event data structure and relevant unit tests (#3955) * Implement event data structure and relevant unit tests * Implement requested changes * Remove unused (useless) file * Remove unnecessary ABC class * Implement tracker methods (#3970) * Implement tracker methods * Improve readability of Event's __repr__ method * Add method to convert Event to dictionary for JSON-compatibility * Send Event metrics via Telemetry (#3979) * Send Event metrics via Telemetry * Add unit tests for event telemetry * Simplify event telemetry unit tests * Clean up unit tests * Add BuildRuntime event and trackers (#3994) * Add BuildRuntime Events * Remove unnecessary file * Prevent tracked Event 'spillage' from previous tests * Fix builder logic * Redefine BuildRuntime to focus on function runtimes * Remove unnecessary file (again) * Add UsedFeature event and trackers (#4003) * Add UsedFeature command and trackers * Remove LocalTest events Events regarding sam local will be implemented at a later time. * Prevent methods from failing if track_event fails * Reformat code to match current standard * Use correct string formatting * Use proper LOG message * Add thread lock to EventTracker (#4019) * Add thread lock to EventTracker * Place lock in appropriate methods * Refactor Event data structure (#4027) * Add new attributes to Event datastructure * Fix non-JSON-serializable field in Event * Fix repr to ensure SQL casting will work * Add comment for timestamp method clarity * Update tests to verify EventTracker thread lock is working (#4035) * Add send_events method to EventTracker (#4039) * Add framework for Event-sending method * Perform some general refactors and cleanup * Implement EventTracker Telemetry unit tests * Cleanup tests and unused imports * Update Metric name for consistency * Repair failing unit test * (Hopefully) repair broken integration test * *Actually* repair broken integration test I had the >= logic backwards lol * Fix integration test For sure this time * Really fix the integration test Windows development is not fun * Properly fix integration test * Reformat files * Implement requested changes * Add Lock logic to send_events method * Implement requested changes * Repair failing tests * Upgrade encryption algorithm from SHA-1 to SHA-256 (#4061) * Upgrade encryption algorithm from SHA1 to SHA256 * Make return value of hash JSON-serializable * Add long event tracker (#4067) * Send events via thread when at capacity (#4072) * Switch send_events to thread when at capacity * Repair failing unit test * Add `sam sync` Events (#4076) * Add sam sync Events * Refactor methods for tracking sync events * Improve SyncFlow tracking * Add additional SyncFlow value * Add workflow events (#4096) * Add workflow_config events * Refactor workflow Events * Add tests for verifying workflow events * Remove redundant (UsedFeature, ESBuild) event * Repair failing unit test * Rename workflow event to prevent ambiguity * Fix send_events logic (#4104) * Make minor tweaks before launch (#4111) * Apply minor changes (some major) * Add session_id for multithreaded send_events * Adjust tests for new multithreading * Fix failing integration test * *Actually* fix failing integration test * Properly fix integration test * This time for sure * Really really repair that test * OK this one SHOULD work I swear * This time for sure * FIX INTEGRATION TEST * IT'S A LIST * Remove unused imports * Add debug log for EventTracker session ID * Place debug log in better position * Fix thread ID bug in EventTracker * Reformat files to standard * Clean up unnecessary files * Add thread ID tracking to infra sync executor * Add missing thread ID to tracked event --------- Co-authored-by: Wing Fung Lau <4760060+hawflau@users.noreply.github.com> Co-authored-by: Leonardo Gama <leogama@amazon.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue(s) does this change fix?
N/A
Why is this change necessary?
With this change, previously implemented Event tracker methods will be able to have their information sent via Telemetry. This provides us with a clear and concise way to actually look at the Event data.
How does it address the issue?
The previously added
EventTrackerclass is imported into the Metric class, allowing access to the Events that are added to it by other methods. Since the decorator function is run after the primary SAM commands are run, it guarantees that any Event information is collected after all commands are executed.What side effects does this change have?
With this new metric in place, the
Metricclass'metricSpecificAttributesfield will have one additional value, "events". This will be a list, containing JSON objects with the following format:Mandatory Checklist
PRs will only be reviewed after checklist is complete
make prpassesmake update-reproducible-reqsif dependencies were changedBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.