Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Putting metrics fails when POWERTOOLS_METRICS_NAMESPACE is not set #1500

Closed
roamingthings opened this issue Nov 7, 2023 · 3 comments · Fixed by #1506
Closed

Putting metrics fails when POWERTOOLS_METRICS_NAMESPACE is not set #1500

roamingthings opened this issue Nov 7, 2023 · 3 comments · Fixed by #1506
Assignees
Labels
bug Something isn't working priority:2 High - core feature or affects 60% of the users

Comments

@roamingthings
Copy link
Contributor

Putting metrics will fail with an software.amazon.cloudwatchlogs.emf.exception.InvalidNamespaceException when environment POWERTOOLS_METRICS_NAMESPACE is not set.

What were you trying to accomplish?

After updating cloudwatchlogs-aws-embedded-metrics to 4.1.1 I started to get InvalidNamespaceException when I try to put a metric for example using

MetricsUtils.withSingleMetric("MyMetric, 1.0, Unit.COUNT, metric ->
...
});

Expected Behavior

Possible behaviors I would expect are:

  • metric is put with a default namespace
  • no exception is thrown (maybe some internal logging). I would not expect a call to a metrics helper to fail my application with an unchecked exception

Current Behavior

software.amazon.cloudwatchlogs.emf.exception.InvalidNamespaceException is thrown which is an unchecked expception.

The reason is that MetricsUtils sets the namespace to an empty string when the environment variable is not set:

    private static String defaultNameSpace() {
        MetricsContext context = MetricsLoggerHelper.metricsContext();
        return "aws-embedded-metrics".equals(context.getNamespace()) ?
                SystemWrapper.getenv("POWERTOOLS_METRICS_NAMESPACE") : context.getNamespace();
    }

Possible Solution

  • Keep the original default namespace (aws-embedded-metrics) if POWERTOOLS_METRICS_NAMESPACE is not set
  • Choose another "default" namespace if POWERTOOLS_METRICS_NAMESPACE is not set

Steps to Reproduce (for bugs)

  1. Call MetricsUtils.withSingleMetric() whithout providing a namespace in POWERTOOLS_METRICS_NAMESPACE

Environment

  • Powertools for AWS Lambda (Java) version used: 1.17.0
  • Packaging format (Layers, Maven/Gradle): Gradle
  • AWS Lambda function runtime: Java 17 (actually discovered in a unit test)
  • Debugging logs
@scottgerring
Copy link
Contributor

scottgerring commented Nov 7, 2023

Hey @roamingthings , thanks for the report! The requirement for the variable is documented, but it does seem a bit odd to explicitly 1/ check if the default hasn't been overriden, and then 2/ override it with an empty string. Inverting the whole thing as you suggest so we override if we have the env var, or use the default otherwise, seems more sensible, and wouldn't be a breaking change.

What do you think @jeromevdl - ?

@jeromevdl
Copy link
Contributor

First, I didn't really understand this code:

private static String defaultNameSpace() {
        MetricsContext context = MetricsLoggerHelper.metricsContext();
        return "aws-embedded-metrics".equals(context.getNamespace()) ?
                SystemWrapper.getenv("POWERTOOLS_METRICS_NAMESPACE") : context.getNamespace();
}

Looks to me that we should do the contrary: if the env var is empty, set a default, as you suggest.

But I'm not sure how the context namespace is actually set. I've found this which makes me think that "aws-embedded-metrics" is the default. So if it is the default, we try to replace it with the env var, otherwise, it means the user already changed the default somehow (how?) and we should use that.

What is missing here is the check on SystemWrapper.getenv("POWERTOOLS_METRICS_NAMESPACE". If null or empty, we should come back to the default.

@jeromevdl jeromevdl added priority:2 High - core feature or affects 60% of the users and removed triage labels Nov 8, 2023
@jeromevdl jeromevdl moved this from Triage to Backlog in Powertools for AWS Lambda (Java) Nov 8, 2023
jeromevdl added a commit that referenced this issue Nov 8, 2023
@jeromevdl jeromevdl moved this from Backlog to Coming soon in Powertools for AWS Lambda (Java) Nov 8, 2023
@jeromevdl jeromevdl reopened this Nov 8, 2023
@github-project-automation github-project-automation bot moved this from Coming soon to Working on it in Powertools for AWS Lambda (Java) Nov 8, 2023
@jeromevdl jeromevdl moved this from Working on it to Coming soon in Powertools for AWS Lambda (Java) Nov 8, 2023
@jeromevdl jeromevdl self-assigned this Nov 8, 2023
@scottgerring scottgerring linked a pull request Nov 9, 2023 that will close this issue
6 tasks
Copy link

This is now released under 1.18.0 version!

jasoniharris pushed a commit that referenced this issue Nov 29, 2023
scottgerring added a commit that referenced this issue Dec 5, 2023
* Setting up Kotlin environment. Converting test to Kotlin.

* Deploying via SAM successfully.

* Added Kotlin example.

* Removing unused Gradle build file.

* Adding SAM template so can be used as an existing project and Java target compatibility

* Adding SAM template so can be used as an existing project

* Updating guidance to use SAM for build and deploy

* Restructuring separate Java and Kotlin examples.

* Updating core examples readme to represent new structure for Java and Kotlin examples.

* Refactoring application code for efficiency, updating build to cover tests too and is more idiomatic and readme to be more descriptive

* Updating to fix trailing \n

* Updating guidance to be more specific for examples

* Adopting new mechanism for specifying jvm target.

* accommodating new project structure

* Fixing link typo after refactoring

* Setting up Kotlin environment. Converting test to Kotlin.

* Deploying via SAM successfully.

* Added Kotlin example.

* Removing unused Gradle build file.

* Adding SAM template so can be used as an existing project and Java target compatibility

* Adding SAM template so can be used as an existing project

* Updating guidance to use SAM for build and deploy

* Restructuring separate Java and Kotlin examples.

* Updating core examples readme to represent new structure for Java and Kotlin examples.

* Refactoring application code for efficiency, updating build to cover tests too and is more idiomatic and readme to be more descriptive

* Updating to fix trailing \n

* Updating guidance to be more specific for examples

* Adopting new mechanism for specifying jvm target.

* accommodating new project structure

* Fixing link typo after refactoring

* Flattening structure back to original to make merging easier for v2

* Adding build for Kotlin Gradle

* Adding build for Kotlin Gradle - Restructuring Java examples to v1 approach

* Correcting paths

* Adding SNAPSHOT support and local capability for Maven. Testing using Java 1.8

* Reviewed and updated against PR comments.

* Un-commenting examples

* Adding validation step for IaC SAM

* Adding Terraform for Java projects IaC validator and linter

* Adding additional projects for SAM validation and matrix approach

* Refactoring stream function to process input logging example with a Lambda Function URL instead of APIGW.

* Demonstrating Java streaming response

* Refactoring stream function to process input logging example to return

* Update CONTRIBUTING.md

* fix: get trace id from system property when env var is not set (#1503)

* fix: check if XRAY Trace ID is present in System property

* chore: remove erroneous extra char in tests

* fix #1500 (#1506)

* feat: Add support for POWERTOOLS_LOGGER_LOG_EVENT (#1510)

* chore: Addition of Warn Message If Invalid Annotation Key While Tracing #1511 (#1512)

* feat: ALC (#1514)

* handle AWS_LAMBDA_LOG configuration

* ALC documentation + code review

* update doc

* chore:Prep release 1.18.0 (#1515)

* chore:prep release 1.18.0

* update version

* update version in kotlin example

* maven local repo in gradle example

* update changelog

---------

Co-authored-by: scottgerring <scottgerring@users.noreply.github.com>

* chore: update version to next snapshot: 1-19.0-SNAPSHOT (#1516)

* update version to next snapshot: 1-19.0-SNAPSHOT

* update version to next snapshot: 1-19.0-SNAPSHOT

* update version to next snapshot: 1-19.0-SNAPSHOT

* building only for LTS

* Add some more margin to the test pause (#1518)

* test: e2e tests with java 21 (#1517)

* e2e tests with java 21

* Run Java21 tests using the Java17 compiler

* Run all of the E2E tests in parallel, not just the first 3

* Try again

* .

* Let's try again

* Add some comment on Java21 to the repo

* Add caveat about lambda runtimes

* Clean up wording a little

---------

Co-authored-by: Scott Gerring <gerrings@amazon.com>

* update doc for ALC (#1520)

* chore: Testing java21 aspectj pre-release (#1519)

* e2e tests with java 21

* use aspectj 1.9.21-SNAPSHOT

* Fix log4j2.xml missing in logging test for java21

* rollback double runtime

* remove comment

* keep aspectj 1.9.7 in parent for java8 compatibility

* use M1 instead of snapshot

* update documentation for aspectj

* update documentation for aspectj

---------

Co-authored-by: Jerome Van Der Linden <jeromevdl@gmail.com>

* chore: Remove build cruft

* Adding context for using RequestStreamHandler

* removing pr_lint

* Update examples/powertools-examples-core/sam/src/main/java/helloworld/AppStream.java

Clarify usage of RequestStreamHandler.

Co-authored-by: Alexey Soshin <alexey.soshin@gmail.com>

---------

Co-authored-by: Jason Harris <harrzjas@amazon.com>
Co-authored-by: Scott Gerring <scottgerring@users.noreply.github.com>
Co-authored-by: Jason Harris <harrzjas@amazon.co.uk>
Co-authored-by: Jérôme Van Der Linden <117538+jeromevdl@users.noreply.github.com>
Co-authored-by: Michele Ricciardi <mriccia@amazon.com>
Co-authored-by: Alexey Soshin <alexey.soshin@gmail.com>
Co-authored-by: jdoherty <jdoherty07@gmail.com>
Co-authored-by: Scott Gerring <gerrings@amazon.com>
Co-authored-by: Jerome Van Der Linden <jeromevdl@gmail.com>
scottgerring added a commit that referenced this issue Feb 22, 2024
* docs: HelloWorldStreamFunction in examples fails with sam (#1532)

* Setting up Kotlin environment. Converting test to Kotlin.

* Deploying via SAM successfully.

* Added Kotlin example.

* Removing unused Gradle build file.

* Adding SAM template so can be used as an existing project and Java target compatibility

* Adding SAM template so can be used as an existing project

* Updating guidance to use SAM for build and deploy

* Restructuring separate Java and Kotlin examples.

* Updating core examples readme to represent new structure for Java and Kotlin examples.

* Refactoring application code for efficiency, updating build to cover tests too and is more idiomatic and readme to be more descriptive

* Updating to fix trailing \n

* Updating guidance to be more specific for examples

* Adopting new mechanism for specifying jvm target.

* accommodating new project structure

* Fixing link typo after refactoring

* Setting up Kotlin environment. Converting test to Kotlin.

* Deploying via SAM successfully.

* Added Kotlin example.

* Removing unused Gradle build file.

* Adding SAM template so can be used as an existing project and Java target compatibility

* Adding SAM template so can be used as an existing project

* Updating guidance to use SAM for build and deploy

* Restructuring separate Java and Kotlin examples.

* Updating core examples readme to represent new structure for Java and Kotlin examples.

* Refactoring application code for efficiency, updating build to cover tests too and is more idiomatic and readme to be more descriptive

* Updating to fix trailing \n

* Updating guidance to be more specific for examples

* Adopting new mechanism for specifying jvm target.

* accommodating new project structure

* Fixing link typo after refactoring

* Flattening structure back to original to make merging easier for v2

* Adding build for Kotlin Gradle

* Adding build for Kotlin Gradle - Restructuring Java examples to v1 approach

* Correcting paths

* Adding SNAPSHOT support and local capability for Maven. Testing using Java 1.8

* Reviewed and updated against PR comments.

* Un-commenting examples

* Adding validation step for IaC SAM

* Adding Terraform for Java projects IaC validator and linter

* Adding additional projects for SAM validation and matrix approach

* Refactoring stream function to process input logging example with a Lambda Function URL instead of APIGW.

* Demonstrating Java streaming response

* Refactoring stream function to process input logging example to return

* Update CONTRIBUTING.md

* fix: get trace id from system property when env var is not set (#1503)

* fix: check if XRAY Trace ID is present in System property

* chore: remove erroneous extra char in tests

* fix #1500 (#1506)

* feat: Add support for POWERTOOLS_LOGGER_LOG_EVENT (#1510)

* chore: Addition of Warn Message If Invalid Annotation Key While Tracing #1511 (#1512)

* feat: ALC (#1514)

* handle AWS_LAMBDA_LOG configuration

* ALC documentation + code review

* update doc

* chore:Prep release 1.18.0 (#1515)

* chore:prep release 1.18.0

* update version

* update version in kotlin example

* maven local repo in gradle example

* update changelog

---------

Co-authored-by: scottgerring <scottgerring@users.noreply.github.com>

* chore: update version to next snapshot: 1-19.0-SNAPSHOT (#1516)

* update version to next snapshot: 1-19.0-SNAPSHOT

* update version to next snapshot: 1-19.0-SNAPSHOT

* update version to next snapshot: 1-19.0-SNAPSHOT

* building only for LTS

* Add some more margin to the test pause (#1518)

* test: e2e tests with java 21 (#1517)

* e2e tests with java 21

* Run Java21 tests using the Java17 compiler

* Run all of the E2E tests in parallel, not just the first 3

* Try again

* .

* Let's try again

* Add some comment on Java21 to the repo

* Add caveat about lambda runtimes

* Clean up wording a little

---------

Co-authored-by: Scott Gerring <gerrings@amazon.com>

* update doc for ALC (#1520)

* chore: Testing java21 aspectj pre-release (#1519)

* e2e tests with java 21

* use aspectj 1.9.21-SNAPSHOT

* Fix log4j2.xml missing in logging test for java21

* rollback double runtime

* remove comment

* keep aspectj 1.9.7 in parent for java8 compatibility

* use M1 instead of snapshot

* update documentation for aspectj

* update documentation for aspectj

---------

Co-authored-by: Jerome Van Der Linden <jeromevdl@gmail.com>

* chore: Remove build cruft

* Adding context for using RequestStreamHandler

* removing pr_lint

* Update examples/powertools-examples-core/sam/src/main/java/helloworld/AppStream.java

Clarify usage of RequestStreamHandler.

Co-authored-by: Alexey Soshin <alexey.soshin@gmail.com>

---------

Co-authored-by: Jason Harris <harrzjas@amazon.com>
Co-authored-by: Scott Gerring <scottgerring@users.noreply.github.com>
Co-authored-by: Jason Harris <harrzjas@amazon.co.uk>
Co-authored-by: Jérôme Van Der Linden <117538+jeromevdl@users.noreply.github.com>
Co-authored-by: Michele Ricciardi <mriccia@amazon.com>
Co-authored-by: Alexey Soshin <alexey.soshin@gmail.com>
Co-authored-by: jdoherty <jdoherty07@gmail.com>
Co-authored-by: Scott Gerring <gerrings@amazon.com>
Co-authored-by: Jerome Van Der Linden <jeromevdl@gmail.com>

* deps: bump aspectj to 1.9.21 for jdk21 (#1536)

* chore: SAM and Terraform IaC extracted from pr_build and simplified approach. (#1533)

* SAM and Terraform IaC extracted from pr_build and simplified approach.

* Update .github/workflows/pr_iac_lint.yml

Co-authored-by: Scott Gerring <scottgerring@users.noreply.github.com>

---------

Co-authored-by: Jason Harris <harrzjas@amazon.co.uk>
Co-authored-by: Scott Gerring <scottgerring@users.noreply.github.com>

* chore: Remove empty CDK test (#1542)

* Remove CDK test

* Build core utilities so that sam validator can find them

* Remove CDK test

* Ok just build it all

* Remove the plan ...

* Remove terraform plan from iac lint

---------

Co-authored-by: Jérôme Van Der Linden <117538+jeromevdl@users.noreply.github.com>

* dependabot on v2 branch (#1548)

* chore: remove unecessary creds acquisition (#1572)

* Merge from main

---------

Co-authored-by: Jason Harris <jiharris90@gmail.com>
Co-authored-by: Jason Harris <harrzjas@amazon.com>
Co-authored-by: Jason Harris <harrzjas@amazon.co.uk>
Co-authored-by: Jérôme Van Der Linden <117538+jeromevdl@users.noreply.github.com>
Co-authored-by: Michele Ricciardi <mriccia@amazon.com>
Co-authored-by: Alexey Soshin <alexey.soshin@gmail.com>
Co-authored-by: jdoherty <jdoherty07@gmail.com>
Co-authored-by: Jerome Van Der Linden <jeromevdl@gmail.com>
@jeromevdl jeromevdl moved this from Coming soon to Shipped in Powertools for AWS Lambda (Java) Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working priority:2 High - core feature or affects 60% of the users
Projects
Status: Shipped
Development

Successfully merging a pull request may close this issue.

3 participants