Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attach additional telemetry data to OTEL traces #1519

Merged
merged 41 commits into from
Nov 1, 2024

Conversation

BryanFauble
Copy link
Collaborator

@BryanFauble BryanFauble commented Oct 8, 2024

Problem:

  1. We did not capture the size of the manifest (Bytes) or the version of schematic that produced the trace.
  2. We did not capture the User ID on the trace.
  3. We could not export logs out of schematic when running it as a regular server
  4. I was encoding the environment into the service name

Solution:

  1. Attach the package version to the resource which is applied to all of the traces
  2. Attach the size of the manifest to the span that covers the download of the manifest
  3. Attach the user ID to the trace
  4. Move code out of conftest for sending logs next to the trace setup, allowing the configuration to work for both integration tests and the code when deployed as a server.
  5. Update to use the DEPLOYMENT_ENVIRONMENT attribute to specify where the application is deployed instead of encoding this into the name of the service. This allows for filtering by service and environment separately.

Testing:

  1. I verified the functionality of the exported trace data:
    image

image

image

@thomasyu888
Copy link
Member

Thanks for the quick PR here.

@BryanFauble BryanFauble changed the title Attach package version Attach additional telemetry data to OTEL traces Oct 8, 2024
Copy link
Member

@thomasyu888 thomasyu888 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me so far! Thanks for adding all of this.

Base automatically changed from fds-2386-synapse-refactoring to develop October 24, 2024 21:07
@BryanFauble BryanFauble force-pushed the attach-additional-telemetry-data branch from 0c18e46 to e6edc21 Compare October 24, 2024 22:32
Copy link

Copy link
Member

@thomasyu888 thomasyu888 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥 LGTM! May want to add some documentation on how to export signoz traces and deploy signoz locally, but going to approve here.

@BryanFauble
Copy link
Collaborator Author

🔥 LGTM! May want to add some documentation on how to export signoz traces and deploy signoz locally, but going to approve here.

@thomasyu888 I've added instructions around this in this commit:

4de4b24

My thought is that I do not want folks to have to setup and run SigNoz locally - When I was testing it a while ago it takes a lot of system resources to run all of the components locally. What I wanted to do instead was to let people send their data into the dev instance of SigNoz. That is what the instructions I made above include.

My concerns with this approach:

  1. Each developer will be given a unique client ID/client secret that they will use to export telemetry data. Off-boarding individuals was my concern here as this system is separate from something like Jumpcloud. It's an additional thing to be aware of as developers come and go.
  2. We are given a quota of 1,000 access token that can be requested per month. If we use up all tokens for development, it might mean that schematic running in AWS cannot export it's telemetry data. This was the main reason why I am suggesting in the documentation that developers running locally manually create a token and use it, instead of the oauth2 client credential exchange that the deployed schematic server (Or github actions) would be using.

@BryanFauble BryanFauble merged commit ad7c8f8 into develop Nov 1, 2024
7 checks passed
@BryanFauble BryanFauble deleted the attach-additional-telemetry-data branch November 1, 2024 19:01
GiaJordan pushed a commit that referenced this pull request Nov 8, 2024
* Attach additional telemetry data to OTEL traces
andrewelamb added a commit that referenced this pull request Nov 14, 2024
* add new tests

* add unit tests

* ran black

* Update schematic/models/validate_attribute.py

Co-authored-by: BryanFauble <17128019+BryanFauble@users.noreply.github.com>

* added tests

* Update README.md

* Update README.md

* add unit tests

* run black

* Update README.md

* temp commit

* remove old tests

* [FDS-2386] Synapse entity tracking and code concurrency updates (#1505)

* [FDS-2386] Synapse entity tracking and code concurrency updates

* ran black

* Update CODEOWNERS

* updated data model type rules to include error param

* fix validate type attribute to use msg level param

* added error handling

* run black

* create Node class

* sat up Node class so that nodes with no displayName fields cause an error on creation

* ran black

* ran mypy

* added new configs for CLI tests

* added new manifests for testing CLI commands

* automate manual CLI tests

* ran black

* Update CODEOWNERS

* Update scan_repo.yml

* Update .github/CODEOWNERS

* Update .github/workflows/scan_repo.yml

* Attach additional telemetry data to OTEL traces (#1519)

* Attach additional telemetry data to OTEL traces

* feat: added tracing for cross manifest validation and file name validation  (#1509)

* add tracing for GX validation

* temp commit

* Updating contribution doc to expect squash and merge (#1534)

* [FDS-2491] Integration tests for Schematic API Test plan (#1512)

Integration tests for Schematic API Test plan

* [FDS-2500] Add Integration Tests for: Manifest Validation (#1516)

* Add Integration Tests for: Manifest Validation

* [FDS-2449] Lock `sphinx` version and update `poetry.lock` (#1530)

Also install `typing-extensions` in the build

* manual test files now being saved in manifests folder

* manual test files now being saved in manifests folder

* remove lines to delete json files that were under git control

* ran black

* add try finally blocks to remove created files

* ran black

* add lines to remove created json files

* Update file annotation store process to require filename be present in order to annotate file

* add lines to remove created json files

* Revert "Update file annotation store process to require filename be present in order to annotate file"

This reverts commit f57c718.

* Don't attempt to annotate the table

* add code in finally blocks to reset config to default values, when tests change them

* complete submit manifest command test

* ran black

* add test for bug case

* update test for table tidyness

* remove unused import

* remove etag column if already present when building temp file view

* catch all exceptions to switch to sequential mode

* update test for updated data

* Revert "update test for updated data"

This reverts commit 255e3c0.

* Revert "catch all exceptions to switch to sequential mode"

This reverts commit 68b0b24.

* catch ValueErrors as well

* Updates for integration test failures (#1537)

* Updates for integration test failures, Config file reset and scope changes

* add todos for removing config resets

* [FDS-2525] Authenticated export of telemetry data (#1527)

* Authenticated export of telemetry data, updating to HTTP otel library

* temp reduce tests

* restore tests

* uncomment tests

* redid how files are deleted, manual tests values are set

* ran black

* [SCHEMATIC-157] Make some dependencies required to avoid `schematic CLI` commands from potentially erroring when doing a pip install (#1540)

* Make otel flash non-optional

* Add dependencies as non-optional

* Update pyproject.toml

---------

Co-authored-by: BryanFauble <17128019+BryanFauble@users.noreply.github.com>
Co-authored-by: Jenny V Medina <jenny.medina@sagebase.org>
Co-authored-by: Thomas Yu <thomas.yu@sagebase.org>
Co-authored-by: Lingling <55448354+linglp@users.noreply.github.com>
Co-authored-by: GiaJordan <gianna.jordan@sagebase.org>
andrewelamb added a commit that referenced this pull request Nov 15, 2024
* add new tests

* add unit tests

* ran black

* Update schematic/models/validate_attribute.py

Co-authored-by: BryanFauble <17128019+BryanFauble@users.noreply.github.com>

* added tests

* Update README.md

* Update README.md

* add unit tests

* run black

* Update README.md

* temp commit

* remove old tests

* [FDS-2386] Synapse entity tracking and code concurrency updates (#1505)

* [FDS-2386] Synapse entity tracking and code concurrency updates

* ran black

* Update CODEOWNERS

* updated data model type rules to include error param

* fix validate type attribute to use msg level param

* added error handling

* run black

* create Node class

* sat up Node class so that nodes with no displayName fields cause an error on creation

* ran black

* ran mypy

* added new configs for CLI tests

* added new manifests for testing CLI commands

* automate manual CLI tests

* ran black

* Update CODEOWNERS

* Update scan_repo.yml

* Update .github/CODEOWNERS

* Update .github/workflows/scan_repo.yml

* Attach additional telemetry data to OTEL traces (#1519)

* Attach additional telemetry data to OTEL traces

* feat: added tracing for cross manifest validation and file name validation  (#1509)

* add tracing for GX validation

* temp commit

* Updating contribution doc to expect squash and merge (#1534)

* [FDS-2491] Integration tests for Schematic API Test plan (#1512)

Integration tests for Schematic API Test plan

* [FDS-2500] Add Integration Tests for: Manifest Validation (#1516)

* Add Integration Tests for: Manifest Validation

* [FDS-2449] Lock `sphinx` version and update `poetry.lock` (#1530)

Also install `typing-extensions` in the build

* manual test files now being saved in manifests folder

* manual test files now being saved in manifests folder

* remove lines to delete json files that were under git control

* ran black

* add try finally blocks to remove created files

* ran black

* add lines to remove created json files

* Update file annotation store process to require filename be present in order to annotate file

* add lines to remove created json files

* Revert "Update file annotation store process to require filename be present in order to annotate file"

This reverts commit f57c718.

* Don't attempt to annotate the table

* add code in finally blocks to reset config to default values, when tests change them

* complete submit manifest command test

* ran black

* add test for bug case

* update test for table tidyness

* remove unused import

* remove etag column if already present when building temp file view

* catch all exceptions to switch to sequential mode

* update test for updated data

* Revert "update test for updated data"

This reverts commit 255e3c0.

* Revert "catch all exceptions to switch to sequential mode"

This reverts commit 68b0b24.

* catch ValueErrors as well

* Updates for integration test failures (#1537)

* Updates for integration test failures, Config file reset and scope changes

* add todos for removing config resets

* [FDS-2525] Authenticated export of telemetry data (#1527)

* Authenticated export of telemetry data, updating to HTTP otel library

* temp reduce tests

* restore tests

* uncomment tests

* redid how files are deleted, manual tests values are set

* ran black

* [SCHEMATIC-157] Make some dependencies required to avoid `schematic CLI` commands from potentially erroring when doing a pip install (#1540)

* Make otel flash non-optional

* Add dependencies as non-optional

* Include schematic_api for now (#1547)

* update toml version to 24.11.1

---------

Co-authored-by: BryanFauble <17128019+BryanFauble@users.noreply.github.com>
Co-authored-by: Jenny V Medina <jenny.medina@sagebase.org>
Co-authored-by: Thomas Yu <thomas.yu@sagebase.org>
Co-authored-by: Lingling <55448354+linglp@users.noreply.github.com>
Co-authored-by: GiaJordan <gianna.jordan@sagebase.org>
andrewelamb added a commit that referenced this pull request Dec 16, 2024
* add new tests

* add unit tests

* ran black

* Update schematic/models/validate_attribute.py

Co-authored-by: BryanFauble <17128019+BryanFauble@users.noreply.github.com>

* added tests

* Update README.md

* Update README.md

* add unit tests

* run black

* Update README.md

* temp commit

* remove old tests

* [FDS-2386] Synapse entity tracking and code concurrency updates (#1505)

* [FDS-2386] Synapse entity tracking and code concurrency updates

* ran black

* Update CODEOWNERS

* updated data model type rules to include error param

* fix validate type attribute to use msg level param

* added error handling

* run black

* create Node class

* sat up Node class so that nodes with no displayName fields cause an error on creation

* ran black

* ran mypy

* added new configs for CLI tests

* added new manifests for testing CLI commands

* automate manual CLI tests

* ran black

* Update CODEOWNERS

* Update scan_repo.yml

* Update .github/CODEOWNERS

* Update .github/workflows/scan_repo.yml

* Attach additional telemetry data to OTEL traces (#1519)

* Attach additional telemetry data to OTEL traces

* feat: added tracing for cross manifest validation and file name validation  (#1509)

* add tracing for GX validation

* temp commit

* Updating contribution doc to expect squash and merge (#1534)

* [FDS-2491] Integration tests for Schematic API Test plan (#1512)

Integration tests for Schematic API Test plan

* [FDS-2500] Add Integration Tests for: Manifest Validation (#1516)

* Add Integration Tests for: Manifest Validation

* [FDS-2449] Lock `sphinx` version and update `poetry.lock` (#1530)

Also install `typing-extensions` in the build

* manual test files now being saved in manifests folder

* manual test files now being saved in manifests folder

* remove lines to delete json files that were under git control

* ran black

* add try finally blocks to remove created files

* ran black

* add lines to remove created json files

* Update file annotation store process to require filename be present in order to annotate file

* add lines to remove created json files

* Revert "Update file annotation store process to require filename be present in order to annotate file"

This reverts commit f57c718.

* Don't attempt to annotate the table

* add code in finally blocks to reset config to default values, when tests change them

* complete submit manifest command test

* ran black

* add test for bug case

* update test for table tidyness

* remove unused import

* remove etag column if already present when building temp file view

* catch all exceptions to switch to sequential mode

* update test for updated data

* Revert "update test for updated data"

This reverts commit 255e3c0.

* Revert "catch all exceptions to switch to sequential mode"

This reverts commit 68b0b24.

* catch ValueErrors as well

* Updates for integration test failures (#1537)

* Updates for integration test failures, Config file reset and scope changes

* add todos for removing config resets

* [FDS-2525] Authenticated export of telemetry data (#1527)

* Authenticated export of telemetry data, updating to HTTP otel library

* temp reduce tests

* restore tests

* uncomment tests

* redid how files are deleted, manual tests values are set

* ran black

* [SCHEMATIC-157] Make some dependencies required to avoid `schematic CLI` commands from potentially erroring when doing a pip install (#1540)

* Make otel flash non-optional

* Add dependencies as non-optional

* Include schematic_api for now (#1547)

* update toml version to 24.11.1 (#1548)

* [SCHEMATIC-193] Support exporting telemetry data from GH integration test runs (#1550)

* Support exporting telemetry data from GH run via access token retrieved via oauth2

* [SCHEMATIC-30, SCHEMATIC-200] Add version to click cli / use pathlib.Path module for checking cache size (#1542)

* Add version to click cli

* Add version

* Run black

* Reformat

* Fix

* Update schematic/schemas/data_model_parser.py

* Add test for check_synapse_cache_size

* Reformat

* Fix tests

* Remove unused parameter

* Install all-extras for now

* Make otel flash non-optional

* Update dockerfile

* Add dependencies as non-optional

* Update pyproject toml

* Fix trivy issue

* Add service version

* Run black

* Move all utils.general tests into separate folder

* Use pre-commit

* Add updates to contribution doc

* Fix

* Add service version to log provider

---------

Co-authored-by: BryanFauble <17128019+BryanFauble@users.noreply.github.com>

* [SCHEMATIC-212] Prevent traces from being combined (#1552)

* Set instance id in github CI run, uninstrument flask auto during integration test run

* [SCHEMATIC-163] Catch error when manifest is generated and existing one doesn't have `entityId` (#1551)

* adds error handling

* adds unit tests for _get_file_entityIds

* updates error message

* adds entityid check to parent func

* updates docstring

* [SCHEMATIC-183] Use paths from file view for manifest generation (#1529)

source manifest file paths from synapse fileviews at generation

* [SCHEMATIC-214] Wrap pandas functions to support not including `None` with the NA values argument (#1553)

* Wrap pandas functions to support not including `None` with the NA values argument

* Ignore types

* pylint issues

* ordering of ignore

* Add to integration test to cover none in a manifest

* Add additional test for manifest

* [SCHEMATIC-210] Add attribute to nones data model (#1555)

Update example_test_nones.model.csv component and add new invalid manifest with nones

* first commit

* ran black

* add test for validateModelManifest

* [SCHEMATIC-214] change data model and component (#1556)

* add valid values to Patient attributes

* update data model

* add test manifests

* update test for new model

* update test for new valid value

* change test to use new manifests

* remove uneeded test file

* revert file

* revert file

* change tests to use new manifests

* remove uneeded manifests

* ran black

* add tests back in

* ran black

* revert manifest

* Split up valid and errored test as separate testing functions

* Remove unused import

---------

Co-authored-by: Gianna Jordan <61707471+GiaJordan@users.noreply.github.com>
Co-authored-by: Andrew Lamb <andrewelamb@gmail.com>
Co-authored-by: Thomas Yu <thomas.yu@sagebase.org>

* incremented packge version number

* Update publish.yml

* Update test.yml

* Update api_test.yml

* Update pdoc.yml

* Update version.py

* updates publish.yml (#1558) (#1561)

Co-authored-by: Brad Macdonald <52762200+BWMac@users.noreply.github.com>

---------

Co-authored-by: BryanFauble <17128019+BryanFauble@users.noreply.github.com>
Co-authored-by: Jenny V Medina <jenny.medina@sagebase.org>
Co-authored-by: Thomas Yu <thomas.yu@sagebase.org>
Co-authored-by: Lingling <55448354+linglp@users.noreply.github.com>
Co-authored-by: GiaJordan <gianna.jordan@sagebase.org>
Co-authored-by: Brad Macdonald <52762200+BWMac@users.noreply.github.com>
Co-authored-by: Gianna Jordan <61707471+GiaJordan@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants