Ensure force_flush at end of AWS Lambda invocation
#296
Conversation
Deploying logfire-docs with
|
| Latest commit: |
45592ed
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://ffc485b7.logfire-docs.pages.dev |
| Branch Preview URL: | https://alex-lambda.logfire-docs.pages.dev |
Codecov ReportAll modified and coverable lines are covered by tests ✅ 📢 Thoughts on this report? Let us know! |
alexmojaki
changed the title
force_flush at end of AWS Lambda invocation
| # https://github.com/getsentry/sentry-python/blob/eab218c91ae2b894df18751e347fd94972a4fe06/sentry_sdk/integrations/aws_lambda.py#L280-L314 | ||
| # So we just look for the client class in all modules. | ||
| # This feels inefficient but it appears be a tiny fraction of the time `configure` takes anyway. | ||
| for mod in list(sys.modules.values()): |
There was a problem hiding this comment.
Same idea as #277. Added a comment.
logfire/_internal/config.py
Outdated
| @@ -797,6 +813,46 @@ def meter(self) -> metrics.Meter: | |||
| def _initialize_credentials_from_token(self, token: str) -> LogfireCredentials | None: | |||
| return LogfireCredentials.from_token(token, requests.Session(), self.base_url) | |||
|
|
|||
| def _ensure_flush_after_lambda(self): | |||
There was a problem hiding this comment.
How slow is this in the general case?
Is there an env var we can check before looping over all modules?
There was a problem hiding this comment.
| def _ensure_flush_after_lambda(self): | |
| def _ensure_flush_after_aws_lambda(self): |
Just to make it clear what we're talking about.
There was a problem hiding this comment.
How slow is this in the general case?
With this test code:
import time
import pandas
import sqlparse
import sqlalchemy
import sqlmodel
import sqlite3
import httpx
import django
import flask
import fastapi
import fastapi_cli
import uvicorn
import pydantic
import aiohttp
import pymongo
import redis
import logfire
start = time.time()
logfire.configure(
token='foobar',
# This speeds up `configure` by about 50% since it means it doesn't check git
service_version='1.0.0',
)
end = time.time()
print(f'{end - start}')Plus some tweaks in the new method, it prints:
len(list(sys.modules.values()))=1820
ensure_flush_after_lambda took 0.0005262080812826753s
0.025816917419433594
So this adds about 2% to the time that configure takes when a large number of modules have been imported already, something which will usually take much longer (>20x longer than configure in this case).
There was a problem hiding this comment.
Is there an env var we can check before looping over all modules?
Right now, probably. It's very hard to trust that AWS will reliably continue to set these variables in the future.
https://github.com/search?q=repo%3Aaws%2Faws-lambda-python-runtime-interface-client%20AWS_LAMBDA_FUNCTION_NAME&type=code indicates that the lambda runtime client doesn't assume the variables are always there. And indeed they might not be if someone is running the client themselves in an emulator or something, which they encourage.
We already only check it in a background thread now. |
No description provided.