Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(pii): Enable PII scrubbing in logentry.params #2956

Merged
merged 2 commits into from
Jan 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Changelog

## Unreleased

**Bug Fixes**:

- Add automatic PII scrubbing to `logentry.params`. ([#2956](https://github.com/getsentry/relay/pull/2956))

## 24.1.0

**Features**:
Expand Down
4 changes: 4 additions & 0 deletions py/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Changelog

## Unreleased

- Add automatic PII scrubbing to `logentry.params`. ([#2956](https://github.com/getsentry/relay/pull/2956))

## 0.8.41

- This release requires Python 3.9 or later. There are no intentionally breaking changes included in this release, but we stopped testing against Python 3.8.
Expand Down
2 changes: 1 addition & 1 deletion relay-event-schema/src/protocol/logentry.rs
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ pub struct LogEntry {

/// Parameters to be interpolated into the log message. This can be an array of positional
/// parameters as well as a mapping of named arguments to their values.
#[metastructure(bag_size = "medium")]
#[metastructure(bag_size = "medium", pii = "true")]
pub params: Annotated<Value>,

/// Additional arbitrary fields for forwards compatibility.
Expand Down
53 changes: 51 additions & 2 deletions relay-pii/src/processor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -469,10 +469,12 @@ mod tests {
use insta::assert_debug_snapshot;
use relay_event_schema::processor::process_value;
use relay_event_schema::protocol::{
Addr, Breadcrumb, DebugImage, DebugMeta, Event, ExtraValue, Headers, LogEntry,
Addr, Breadcrumb, DebugImage, DebugMeta, Event, ExtraValue, Headers, LogEntry, Message,
NativeDebugImage, Request, Span, TagEntry, Tags, TraceContext,
};
use relay_protocol::{assert_annotated_snapshot, Annotated, FromValue, Object, Value};
use relay_protocol::{
assert_annotated_snapshot, get_value, Annotated, FromValue, Object, Value,
};

use super::*;
use crate::{DataScrubbingConfig, PiiConfig, ReplaceRedaction};
Expand Down Expand Up @@ -1525,4 +1527,51 @@ mod tests {

assert_debug_snapshot!(&data);
}

#[test]
fn test_logentry_params_scrubbed() {
let config = serde_json::from_str::<PiiConfig>(
r##"
{
"applications": {
"$string": ["@anything:remove"]
}
}
"##,
)
.unwrap();

let mut event = Annotated::new(Event {
logentry: Annotated::new(LogEntry {
message: Annotated::new(Message::from("failed to parse report id=%s".to_owned())),
formatted: Annotated::new("failed to parse report id=1".to_string().into()),
params: Annotated::new(Value::Array(vec![Annotated::new(Value::String(
"12345".to_owned(),
))])),
..Default::default()
}),
..Default::default()
});

let mut processor = PiiProcessor::new(config.compiled());
process_value(&mut event, &mut processor, ProcessingState::root()).unwrap();

let params = get_value!(event.logentry.params!);
assert_debug_snapshot!(params, @r#"Array(
[
Meta {
remarks: [
Remark {
ty: Removed,
rule_id: "@anything:remove",
range: None,
},
],
errors: [],
original_length: None,
original_value: None,
},
],
)"#);
}
}