Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add telemetry completeness tests #3650

Merged
merged 6 commits into from
Dec 13, 2024
Merged

Add telemetry completeness tests #3650

merged 6 commits into from
Dec 13, 2024

Conversation

bm1549
Copy link
Contributor

@bm1549 bm1549 commented Dec 9, 2024

Motivation

Adds a new test for telemetry completeness, ensuring what is specified in dd-go is present in our individual libraries

With the current implementation, this tests for telemetry of an empty configuration which may be lossy in some languages

These tests somewhat eliminate the need for equivalent tests in dd-trace-js and tests in dd-trace-dotnet (missing ref)

Note for reviewer: I'm a bit suspicious about the correctness of this line, but it seems to better match what is on this notebook

Changes

Adds a test across languages

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner. We're working on refining the codeowners file quickly.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • If PR title starts with [<language>], double-check that only <language> is impacted by the change
  • No system-tests internal is modified. Otherwise, I have the approval from R&P team
  • CI is green, or failing jobs are not related to this change (and you are 100% sure about this statement)
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added (or removed)?

@bm1549 bm1549 marked this pull request as ready for review December 10, 2024 01:13
@bm1549 bm1549 requested review from a team as code owners December 10, 2024 01:13
@bm1549 bm1549 requested a review from paullegranddc December 10, 2024 01:13
@bm1549 bm1549 requested review from mabdinur and a team as code owners December 10, 2024 02:40
Copy link
Collaborator

@cbeauchesne cbeauchesne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI is failing on the new test

@bm1549 bm1549 requested a review from cbeauchesne December 10, 2024 21:50
@bm1549
Copy link
Contributor Author

bm1549 commented Dec 10, 2024

CI is failing on the new test

@cbeauchesne I've fixed the CI and all failures are now unrelated to the new test. Please re-review 🙏

tests/telemetry_intake/update.sh Show resolved Hide resolved
tests/test_telemetry.py Outdated Show resolved Hide resolved
tests/test_telemetry.py Outdated Show resolved Hide resolved
tests/test_telemetry.py Outdated Show resolved Hide resolved
@cbeauchesne
Copy link
Collaborator

cbeauchesne commented Dec 11, 2024

Could you also update your branch? it'll fix php errors

tests/telemetry_intake/update.sh Show resolved Hide resolved
tests/telemetry_intake/update.sh Show resolved Hide resolved
tests/telemetry_intake/update.sh Show resolved Hide resolved
tests/telemetry_intake/update.sh Outdated Show resolved Hide resolved
tests/telemetry_intake/update.sh Outdated Show resolved Hide resolved
tests/test_telemetry.py Outdated Show resolved Hide resolved
tests/test_telemetry.py Outdated Show resolved Hide resolved
tests/test_telemetry.py Outdated Show resolved Hide resolved
tests/test_telemetry.py Outdated Show resolved Hide resolved
tests/test_telemetry.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@robertomonteromiguel robertomonteromiguel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are test failures related with this PR

elif isinstance(obj, list):
result.extend(obj)
else:
logger.error(f"Unexpected type in concat: {type(obj).__name__}")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this occurs, I expect the test to fail due to missing config keys, so not throwing here

@bm1549 bm1549 force-pushed the bm1549/config-telemetry branch from 5d696a3 to 93a21f1 Compare December 12, 2024 15:06
# If this script deletes the JSON files in system-tests, it's likely this is no longer valid
SUBDIR="trace/apps/tracer-telemetry-intake/telemetry-payload/static/"
# The "main" branch in dd-go
BRANCH="bm1549/more-missing-configs" # This should ALWAYS be "prod" in master. Please revert changes before merging
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending (but not blocked by) https://github.com/DataDog/dd-go/pull/161868

Suggested change
BRANCH="bm1549/more-missing-configs" # This should ALWAYS be "prod" in master. Please revert changes before merging
BRANCH="prod"

Copy link
Collaborator

@cbeauchesne cbeauchesne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lats small bit : if a github team exists, add it as the owner for the tests/telemetry_intake folder

@bm1549 bm1549 merged commit 76ef778 into main Dec 13, 2024
283 checks passed
@bm1549 bm1549 deleted the bm1549/config-telemetry branch December 13, 2024 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants