Replace pattern Issues with Transform processor #34123
Comments
|
Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
|
Could you try enabling debug logging to look at where your data is located inside the payload? Instructions to enable debug logging can be found here: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl#troubleshooting. I suspect the body may not exist at the OTTL path you are specifying, and the regex isn't working because it isn't running on any data. In particular, I'm not clear that this is the right path: |
|
@evan-bradley Thank You very much for your prompt response. I learned that path is body. Turned on debug level and based on collector logs condition matched but its not redacting the value. Also I do noticed issues with conditions as well. Please review and advice. Agent Config: Collector Log: |
|
@evan-bradley Please do review above mentioned findings and provide your inputs. Much appreciated your timely response. |
|
The path you want is just This will only remove the word |
|
This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
|
This issue has been closed as inactive because it has been stale for 120 days with no activity. |
Component(s)
processor/transform
Describe the issue you're reporting
Have requirement to mask fullName value in application logs. Since we are migrating from O11y to Splunk Cloud, thought of implementing the transform process at client side (Splunk Agent). Created RegEx and its working fine in regex101 but some reason its not functioning at processor end. I have not noticed any process errors in agent logs. Here is splunk agent config and sample raw message.
Helm: 0.103.0
Azure AKS Clusters
agent:
enabled: true
securityContext:
runAsUser: 20000
runAsGroup: 20000
config:
processors:
transform:
log_statements:
- context: log
statements:
- replace_pattern(attributes["body.message"], "(."fullName\\\\\\\":\\\\\\\")(.?)(\\\\\\\".*?)", "$1xxx$3")
Tried with below regex as well:
- replace_pattern(attributes["body.message"], "(."fullName\\\":\\\")(.?)(\\\".*?)", "$1xxx$3")
Raw Message:
{"timestamp":"2024-07-16T17:51:27.133135994Z","level":"DEBUG","trace_id":"c06882917a367265fea5042cbce4b632","span_id":"0d31813def1db136","message":"our JSON body: "{\"query\":\"mutation InitiateDeposit__money_movement_microservice__0($input:InitiateDepositInput!){initiateDeposit(input:$input){depositId}}\",\"operationName\":\"InitiateDeposit__money_movement_microservice__0\",\"variables\":{\"input\":{\"accountId\":\"lHmY2QJKoESJyaRfvzVV4DPG6mp18yt3fPsBsoJHoYn7BzSo39yoGw==\",\*"fullName\":\"John Moore\"}}}**"","target":"apollo_router::services::subgraph_service","spans":[{"http.method":"POST","http.request.method":"POST","http.route":"/gateway","http.flavor":"HTTP/1.1","name":"request"},{"http.method":"POST","http.request.method":"POST","http.route":"/gateway","http.flavor":"HTTP/1.1","trace_id":"c06882917a367265fea5042cbce4b632","url.path":"/gateway","client.name":"","client.version":"","name":"router"},{"graphql.document":"mutation InitiateDeposit($input: InitiateDepositInput!) {\n initiateDeposit(input: $input) {\n depositId\n }\n}","graphql.operation.name":"InitiateDeposit","graphql.operation.name":"InitiateDeposit","name":"supergraph"},{"graphql.operation.type":"mutation","name":"execution"},{"apollo.subgraph.name":"money-movement-microservice","name":"fetch"},{"apollo.subgraph.name":"money-movement-microservice","graphql.document":"mutation InitiateDeposit__money_movement_microservice__0($input:InitiateDepositInput!){initiateDeposit(input:$input){depositId}}","graphql.operation.name":"InitiateDeposit__money_movement_microservice__0","subgraph.name":"money-movement-microservice","name":"subgraph"}],"resource":{"deployment.environment":"dev-qa","service.name":"***-federated-gateway","service.version":"1.45.1","process.executable.name":"router"}}
The expected message in Splunk logs should be masked.
{timestamp":"2024-07-16T17:51:27.133135994Z","level":"DEBUG","trace_id":"c06882917a367265fea5042cbce4b632","span_id":"0d31813def1db136","message":"our JSON body: "{\"query\":\"mutation InitiateDeposit__money_movement_microservice__0($input:InitiateDepositInput!){initiateDeposit(input:$input){depositId}}\",\"operationName\":\"InitiateDeposit__money_movement_microservice__0\",\"variables\":{\"input\":{\"accountId\":\"lHmY2QJKoESJyaRfvzVV4DPG6mp18yt3fPsBsoJHoYn7BzSo39yoGw==\",\"fullName\":\"xxx\"}}}","target":"apollo_router::services::subgraph_service","spans":[{"http.method":"POST","http.request.method":"POST","http.route":"/gateway","http.flavor":"HTTP/1.1","name":"request"},{"http.method":"POST","http.request.method":"POST","http.route":"/gateway","http.flavor":"HTTP/1.1","trace_id":"c06882917a367265fea5042cbce4b632","url.path":"/gateway","client.name":"","client.version":"","name":"router"},{"graphql.document":"mutation InitiateDeposit($input: InitiateDepositInput!) {\n initiateDeposit(input: $input) {\n depositId\n }\n}","graphql.operation.name":"InitiateDeposit","graphql.operation.name":"InitiateDeposit","name":"supergraph"},{"graphql.operation.type":"mutation","name":"execution"},{"apollo.subgraph.name":"money-movement-microservice","name":"fetch"},{"apollo.subgraph.name":"money-movement-microservice","graphql.document":"mutation InitiateDeposit__money_movement_microservice__0($input:InitiateDepositInput!){initiateDeposit(input:$input){depositId}}","graphql.operation.name":"InitiateDeposit__money_movement_microservice__0","subgraph.name":"money-movement-microservice","name":"subgraph"}],"resource":{"deployment.environment":"dev-qa","service.name":"***-federated-gateway","service.version":"1.45.1","process.executable.name":"router"}}
Thank You in Advance.
The text was updated successfully, but these errors were encountered: