Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lambda fails due to timeout with extensionOverhead #1011

Open
Dreamescaper opened this issue Dec 6, 2024 · 0 comments
Open

Lambda fails due to timeout with extensionOverhead #1011

Dreamescaper opened this issue Dec 6, 2024 · 0 comments

Comments

@Dreamescaper
Copy link

Dreamescaper commented Dec 6, 2024

Describe the bug
We have a lambda with a relatively heavy load, and and it fails from time to time with timeout.
At the start of execution, I see the following error log:

{
    "level": "error",
    "ts": 1733502870.3059413,
    "logger": "telemetryAPI.Listener",
    "msg": "Unexpected stop on HTTP Server",
    "error": "listen tcp 169.254.100.6:53612: bind: address already in use"
}

And then after the execution it simply fails with timeout:

{
    "time": "2024-12-06T16:35:01.725Z",
    "type": "platform.report",
    "record": {
        "requestId": "8051dde8-d1c4-42c0-9e35-fb814bf129e6",
        "metrics": {
            "durationMs": 30000,
            "billedDurationMs": 30000,
            "memorySizeMB": 1024,
            "maxMemoryUsedMB": 171
        },
        "tracing": {
            "spanId": "79aaa60f1ecbf91b",
            "type": "X-Amzn-Trace-Id",
            "value": "Root=1-67532795-a2d3d266ff0212fdf81b0489;Parent=74bbcb11b6a8a88b;Sampled=1;Lineage=1:f84e2b95:0"
        },
        "spans": [
            {
                "name": "extensionOverhead",
                "start": "2024-12-06T16:34:31.529Z",
                "durationMs": 28176.889
            }
        ],
        "status": "timeout"
    }
}

Steps to reproduce
I don't know how to reproduce it, as the error happens couple of times per day (when this lambda is invoked ~5 times per second).

What did you expect to see?
No errors, no timeout.

What did you see instead?
Timeout failure

What version of collector/language SDK version did you use?
Version: aws-otel-collector-amd64-ver-0-102-1

What language layer did you use?
Config: .NET 8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant