Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue to send log from Google Cloud Run #554

Open
m-lce opened this issue Oct 29, 2024 · 3 comments
Open

Issue to send log from Google Cloud Run #554

m-lce opened this issue Oct 29, 2024 · 3 comments
Labels
documentation Improvements or additions to documentation Feature Request help wanted Extra attention is needed

Comments

@m-lce
Copy link

m-lce commented Oct 29, 2024

Description

I instrument a FastAPI server running in Google Cloud Run.
Locally it works perferctly but with the same Docker image in Google Cloud Run some messages did not reach the Logfire backend and there is no errors in the logs.
Do you know what could be the issue

Python, Logfire & OS Versions, related packages (not required)

logfire="1.3.2"
platform="Linux-6.11.5-arch1-1-x86_64-with-glibc2.35"
python="3.12.7 (main, Oct 29 2024, 11:47:05) [GCC 11.4.0]"
[related_packages]
requests="2.32.3"
pydantic="2.9.2"
fastapi="0.115.4"
openai="1.52.2"
protobuf="4.25.5"
rich="13.9.3"
executing="2.1.0"
opentelemetry-api="1.27.0"
opentelemetry-exporter-otlp="1.27.0"
opentelemetry-exporter-otlp-proto-common="1.27.0"
opentelemetry-exporter-otlp-proto-grpc="1.27.0"
opentelemetry-exporter-otlp-proto-http="1.27.0"
opentelemetry-instrumentation="0.48b0"
opentelemetry-instrumentation-aiohttp-client="0.48b0"
opentelemetry-instrumentation-asgi="0.48b0"
opentelemetry-instrumentation-dbapi="0.48b0"
opentelemetry-instrumentation-fastapi="0.48b0"
opentelemetry-instrumentation-grpc="0.48b0"
opentelemetry-instrumentation-httpx="0.48b0"
opentelemetry-instrumentation-jinja2="0.48b0"
opentelemetry-instrumentation-pymongo="0.48b0"
opentelemetry-instrumentation-pymysql="0.48b0"
opentelemetry-instrumentation-requests="0.48b0"
opentelemetry-instrumentation-sqlite3="0.48b0"
opentelemetry-instrumentation-starlette="0.48b0"
opentelemetry-instrumentation-urllib="0.48b0"
opentelemetry-instrumentation-urllib3="0.48b0"
opentelemetry-proto="1.27.0"
opentelemetry-sdk="1.27.0"
opentelemetry-semantic-conventions="0.48b0"
opentelemetry-util-http="0.48b0"
@m-lce m-lce added the bug Bug related to the Logfire Python SDK label Oct 29, 2024
@m-lce m-lce closed this as completed Oct 29, 2024
@m-lce m-lce closed this as not planned Won't fix, can't repro, duplicate, stale Oct 29, 2024
@m-lce m-lce reopened this Oct 29, 2024
@alexmojaki
Copy link
Contributor

Google Cloud Run is sending a traceparent header that OpenTelemetry picks up automatically, and it's also sampling traces. Here's some code to prevent that by effectively disabling distributed tracing:

from opentelemetry.context import get_current
from opentelemetry.propagate import set_global_textmap
from opentelemetry.propagators.textmap import TextMapPropagator


class NullPropagator(TextMapPropagator):
    def extract(self, *args, **kwargs):
        return get_current()

    def inject(self, *args, **kwargs):
        pass

    @property
    def fields(self):
        return set()


set_global_textmap(NullPropagator())

@alexmojaki
Copy link
Contributor

Things we need to maybe do:

  1. Document this recipe.
  2. Add a feature to the SDK to make it easier.
  3. Automatically detect the traceparent header or something and warn the user or something, since this kind of unintentional distributed tracing seems quite common.
  4. Pressure Cloud Run to do better here, e.g. allow setting the sample rate or disabling the traceparent header. Related: Not all traces are getting to GCP GoogleCloudPlatform/opentelemetry-cloud-run#18
  5. Ask OTEL for ideas.

Currently if you do want distributed tracing based on a traceparent header sent from somewhere other than Cloud Run, we don't have a good solution. You can forcibly turn off sampling (generously assuming you don't want your own sampling) but missing a root span is not good anyway.

@alexmojaki alexmojaki added help wanted Extra attention is needed Feature Request and removed bug Bug related to the Logfire Python SDK labels Oct 29, 2024
@m-lce
Copy link
Author

m-lce commented Oct 29, 2024

It works perfectly now !
Thank you very much for the quick answer !

@sydney-runkle sydney-runkle added the documentation Improvements or additions to documentation label Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation Feature Request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants