Running dotnet-monitor in a docker sidecar (not root)

[jamescarter-le]

Documentation Request

Running dotnet-monitor in a docker sidecar container mode, when using non-root users (default for dotnet-monitor, and aspnet images).

I am adding dotnet-monitor to AWS ECS container, as a side-car. I'm unable to get dotnet-monitor to read the socket file from the app when running both as their default users. I understand that the default UID of aspnet and (I suppose) dotnet-monitor is 1000 (app).

This gives me Permission Denied from IpcSocket. Running them both as root does solve this problem, however I would like to drop down to the default permission set as recommend by Microsoft and the default containers.

I attach a normal Docker volume (non persistent) to both containers, and do not specify their UID when starting them. This gives me permission denied.
I've tried setting both their users to 'app', '1000', '1654', but none of these settings work.

What am I missing here? I feel like a little more documentation on this (mostly likely the default?) configuration would be very useful.

Previous documentation

Existing docker compose for this scenario

Configuration example

{
    "taskDefinitionArn": "arn:aws:ecs:eu-west-1:myaws:task-definition/MyContainerTask:30",
    "containerDefinitions": [
        {
            "name": "MyContainer",
            "image": "myaws.dkr.ecr.eu-west-1.amazonaws.com/mycontainer:latest",
            "cpu": 0,
            "memory": 448,
            "links": [],
            "portMappings": [
                {
                    "name": "http",
                    "containerPort": 8080,
                    "hostPort": 0,
                    "protocol": "tcp",
                    "appProtocol": "http"
                }
            ],
            "essential": true,
            "entryPoint": [],
            "command": [],
            "environment": [
                {
                    "name": "OTEL_ENDPOINT",
                    "value": "http://otel-collector.sandbox.local:4317"
                },
                {
                    "name": "DOTNET_DiagnosticPorts",
                    "value": "/diag/dotnet-monitor.sock,nosuspend"
                },
                {
                    "name": "DOTNET_gcServer",
                    "value": "1"
                }
            ],
            "environmentFiles": [],
            "mountPoints": [
                {
                    "sourceVolume": "diagnostics",
                    "containerPath": "/diag",
                    "readOnly": false
                }
            ],
            "volumesFrom": [],
            "secrets": [],
            "dnsServers": [],
            "dnsSearchDomains": [],
            "extraHosts": [],
            "dockerSecurityOptions": [],
            "dockerLabels": {},
            "ulimits": [],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "logs-from-ecs",
                    "mode": "non-blocking",
                    "awslogs-multiline-pattern": "^(trace|debug|info|warn|error|critical|none)",
                    "awslogs-region": "eu-west-1",
                    "awslogs-stream-prefix": "containers"
                },
                "secretOptions": []
            },
            "systemControls": [],
            "credentialSpecs": []
        },
        {
            "name": "dotnet-monitor",
            "image": "mcr.microsoft.com/dotnet/monitor:9",
            "cpu": 256,
            "memory": 512,
            "links": [],
            "portMappings": [
                {
                    "containerPort": 52323,
                    "hostPort": 0,
                    "protocol": "tcp"
                }
            ],
            "essential": false,
            "entryPoint": [],
            "command": [],
            "environment": [
                {
                    "name": "DOTNETMONITOR_Urls",
                    "value": "http://+:52323"
                },
                {
                    "name": "DOTNETMONITOR_DiagnosticPort__ConnectionMode",
                    "value": "Listen"
                },
                {
                    "name": "DOTNETMONITOR_Storage__DumpTempFolder",
                    "value": "/diag/dumps"
                },
                {
                    "name": "DOTNETMONITOR_DiagnosticPort__EndpointName",
                    "value": "/diag/dotnet-monitor.sock"
                }
            ],
            "environmentFiles": [],
            "mountPoints": [
                {
                    "sourceVolume": "diagnostics",
                    "containerPath": "/diag",
                    "readOnly": false
                }
            ],
            "volumesFrom": [],
            "secrets": [
                {
                    "name": "Authentication__MonitorApiKey__Subject",
                    "valueFrom": "arn:aws:secretsmanager:eu-west-1:myaws:secret:DotnetMonitorAuthentication:Subject::"
                },
                {
                    "name": "Authentication__MonitorApiKey__PublicKey",
                    "valueFrom": "arn:aws:secretsmanager:eu-west-1:myaws:secret:DotnetMonitorAuthentication:PublicKey::"
                }
            ],
            "dnsServers": [],
            "dnsSearchDomains": [],
            "extraHosts": [],
            "dockerSecurityOptions": [],
            "dockerLabels": {},
            "ulimits": [],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "logs-from-ecs",
                    "mode": "non-blocking",
                    "awslogs-region": "eu-west-1",
                    "awslogs-stream-prefix": "containers"
                },
                "secretOptions": []
            },
            "systemControls": [],
            "credentialSpecs": []
        }
    ],
    "family": "MyContainerTask",
    "taskRoleArn": "arn:aws:iam::myaws:role/Exec-0GqZn4YCDiZp",
    "executionRoleArn": "arn:aws:iam::myaws:role/Exec-ApgSEYISB08l",
    "networkMode": "bridge",
    "revision": 30,
    "volumes": [
        {
            "name": "diagnostics",
            "host": {}
        }
    ],
    "status": "ACTIVE",
    "requiresAttributes": [
        {
            "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
        },
        {
            "name": "ecs.capability.execution-role-awslogs"
        },
        {
            "name": "com.amazonaws.ecs.capability.ecr-auth"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
        },
        {
            "name": "ecs.capability.secrets.asm.environment-variables"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.17"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.28"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.30"
        },
        {
            "name": "com.amazonaws.ecs.capability.task-iam-role"
        },
        {
            "name": "ecs.capability.execution-role-ecr-pull"
        },
        {
            "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
        }
    ],
    "placementConstraints": [],
    "compatibilities": [
        "EC2"
    ],
    "requiresCompatibilities": [
        "EC2"
    ],
    "registeredAt": "2024-11-28T15:38:24.620Z",
    "registeredBy": "",
    "tags": []
}

[lbussell]

Related: dotnet/dotnet-docker#6081

[jander-msft]

I don't have a lot of knowledge of ECS, but I can provide some ideas to look at:

• The default user has an ID of 1654 starting with .NET Monitor 8 and going forward. Prior to that version, it was different for each tag based on the base image, but since those are out of support at this point, I won't go into details there.
• Based on the symptoms you've described (accessing Unix domain socket works with root, but not for non-root), it seems like only root is able to use the diagnostics volume. You may need to specify dockerVolumeConfiguration on the diagnostics volume such that it allows the 1654 user to read and write from it.