Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ref(normalization): Replace LightNormalization with NormalizeProcessor #2731

Merged
merged 4 commits into from
Nov 17, 2023
Merged

ref(normalization): Replace LightNormalization with NormalizeProcessor #2731

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c7e88fc
Extend `NormalizeProcessorConfig`
iker-barriocanal Nov 16, 2023
e5b9a35
Remove light normalization
iker-barriocanal Nov 16, 2023
7ffc1a4
Replace lightnormalization calls with normalizeprocessor
iker-barriocanal Nov 16, 2023
bd64854
fix docs
iker-barriocanal Nov 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions relay-cabi/src/processing.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ use once_cell::sync::OnceCell;
use relay_common::glob::{glob_match_bytes, GlobOptions};
use relay_dynamic_config::{normalize_json, validate_json, GlobalConfig, ProjectConfig};
use relay_event_normalization::{
light_normalize_event, GeoIpLookup, LightNormalizationConfig, RawUserAgentInfo, StoreConfig,
GeoIpLookup, NormalizeProcessor, NormalizeProcessorConfig, RawUserAgentInfo, StoreConfig,
StoreProcessor,
};
use relay_event_schema::processor::{process_value, split_chunks, ProcessingState};
Expand Down Expand Up @@ -112,7 +112,7 @@ pub unsafe extern "C" fn relay_store_normalizer_normalize_event(
let processor = normalizer as *mut StoreProcessor;
let mut event = Annotated::<Event>::from_json((*event).as_str())?;
let config = (*processor).config();
let light_normalization_config = LightNormalizationConfig {
let normalization_config = NormalizeProcessorConfig {
client_ip: config.client_ip.as_ref(),
user_agent: RawUserAgentInfo {
user_agent: config.user_agent.as_deref(),
Expand All @@ -137,7 +137,11 @@ pub unsafe extern "C" fn relay_store_normalizer_normalize_event(
enable_trimming: config.enable_trimming.unwrap_or_default(),
measurements: None,
};
light_normalize_event(&mut event, light_normalization_config)?;
process_value(
&mut event,
&mut NormalizeProcessor::new(normalization_config),
ProcessingState::root(),
)?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we still use store processor here after the new NormalizeProcessor?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We still need to run both processors in some circumstances until we unify them.

process_value(&mut event, &mut *processor, ProcessingState::root())?;
RelayStr::from_string(event.to_json()?)
}
Expand Down
2 changes: 1 addition & 1 deletion relay-event-normalization/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ pub struct StoreConfig {

/// The processor that normalizes events for processing and storage.
///
/// This processor is a superset of [`light_normalize_event`], that runs additional and heavier
/// This processor is a superset of [`NormalizeProcessor`], that runs additional and heavier
/// normalization steps. These normalizations should ideally be performed on events that are likely
/// to be ingested, after other functionality such as inbound filters have run.
///
Expand Down
Loading
Loading