Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add Metrics middleware using CloudWatch EMF #15

Merged
merged 18 commits into from
Apr 9, 2020
Merged

Conversation

heitorlessa
Copy link
Contributor

@heitorlessa heitorlessa commented Apr 8, 2020

Issue #, if available: #1

Description of changes:

  • Create quick script to fetch env vars
  • Rename Lambda functions/resources that cannot accept a Log Group parameter
    • Payment
    • Loyalty
    • Booking
  • Explicitly create log groups with expiration
  • Extend templates to accept log expiration date
  • Enable API Gateway access logs
  • Create API Gateway IAM Role to ship logs
    • No longer needed
  • EMF: Allow creation of multiple metrics at the end of a function
  • EMF: Allow creation of a metric with a different dimension
  • EMF: Move Metric Manager into a separate file
  • Tests
    • Test Single metric
    • Test Single metric only reports one metric
    • Test multiple metrics
    • Test schema invalidation error
    • Test incorrect Metric unit
    • Test incorrect Metric value
    • Test multiple namespaces
    • Test no metrics
    • Test log_metrics decorator
    • Test log_metrics decorator as standalone (call the decorated fn)
    • Test namespace env var
    • Test log_metric to ensure it raises deprecation error
    • Update log_metric tests
    • Test over 100 metrics for two separate EMF objects
    • Test maximum of dimensions
    • Double check if an EMF object can surpass 1 Mb
  • Add deprecation warning for log_metric
  • Add initial docs
  • Update HISTORY
  • Update example

Features

  • Create up to 100 metrics using a single EMF object (large JSON blob)
  • Create an one off metric with a different dimension (e.g. Cold Start metric using function_version while all other metrics use service_name as a dimension)
  • Validate against common mistakes that CloudWatch would otherwise fail silently
  • No stack, custom resource, data collection needed — Metrics are created async by CloudWatch EMF

UX

Multiple metrics

Used alongside other middlewares that call the decorated function

from lambda_python_powertools.metrics import Metrics, MetricUnit

metrics = Metrics()
metrics.add_namespace(name="ServerlessAirline")
metrics.add_metric(name="ColdStart", unit="Count", value=1)
metrics.add_dimension(name="service", value="booking")

@metrics.log_metrics
@tracer.capture_lambda_handler
def testing():
    metrics.add_metric(name="BookingConfirmation", unit="Count", value=1)
    some_code()
    return True

def some_code():
    metrics.add_metric(name="some_other_metric", unit=MetricUnit.Seconds, value=1)
    ...

Single metric feature

Context manager with parameters using similar DX as multiple metrics

with single_metric(name="ColdStart", unit=MetricUnit.Count, value=1) as metric:
    metric.add_dimension(name="function_context", value="$LATEST")

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@heitorlessa heitorlessa self-assigned this Apr 8, 2020
Copy link
Contributor

@jfuss jfuss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you test that producing the whl/sdist with poetry build still included all the files you need (for example is the .json files included correctly in the package)?

python/README.md Outdated Show resolved Hide resolved
python/aws_lambda_powertools/metrics/base.py Show resolved Hide resolved
python/aws_lambda_powertools/metrics/base.py Show resolved Hide resolved
python/aws_lambda_powertools/metrics/base.py Outdated Show resolved Hide resolved
python/aws_lambda_powertools/metrics/base.py Show resolved Hide resolved
python/aws_lambda_powertools/metrics/metric.py Outdated Show resolved Hide resolved
@heitorlessa
Copy link
Contributor Author

heitorlessa commented Apr 9, 2020

@marcioemiranda What are your thoughts? I know you're using the lib extensively

@heitorlessa heitorlessa merged commit a9fc2bd into develop Apr 9, 2020
@heitorlessa heitorlessa deleted the feat/metrics branch April 9, 2020 15:40
@heitorlessa heitorlessa added feature New feature or functionality and removed enhancement labels Jun 3, 2020
@heitorlessa heitorlessa changed the title Feature - EMF Metrics middleware feat: add Metrics middleware using CloudWatch EMF Jun 3, 2020
heitorlessa referenced this pull request in heitorlessa/aws-lambda-powertools-python Jun 17, 2020
* feat: add metrics middleware

* fix(Makefile): increase verbosity for test results

* chore: linting

* chore: add deprecation warning log_metric

* docs: add initial docs for metrics

* chore: version bump

* fix: replace jsonschema, automate wheels for Linux

* feat: add docs generation

* feat(Makefile): add docs target

* docs: update to reflect metrics middleware, beta

* docs: update history

* improv(metrics): address feedbacks from Jacob

* fix(metrics): correct decorator logic

* chore: version bump

* fix(Makefile): fix build linux wheels target

* fix(metrics): lambda handler return and exceptions

* chore: version bump

* improv: update example to use new metrics feat

Co-authored-by: heitorlessa <lessa@amazon.co.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants