ref(replays): increase chunking limit to 15MB (#2032)

We were previously chunking kafka messages on the replay recording topic
to 1 MiB. We are aiming to eliminate this chunking as it requires us to
maintain a redis cluster / constrains us to in-order processing. Other
infra at Sentry has successfully experimented with larger kafka
messages, so we feel this is safe.

With #2031 this should pretty
much mean we no longer chunk any messages. We can remove the chunking
logic once this is confirmed after deploy.

- [x] Increase chunking limit from 1MiB to 15 MiB. 

We chose 15MiB as the limit, as we're eventually going to combine the
replay event with the replay recording, and will want some extra
headroom for this in addition to our 10 MiB recording limit.

Depends on ops increasing our kafka payload size limit first.

CHANGELOG.md

@@ -11,6 +11,7 @@
 - Include unknown feature flags in project config when serializing it. ([#2040](https://github.com/getsentry/relay/pull/2040))
 - Copy transaction tags to the profile. ([#1982](https://github.com/getsentry/relay/pull/1982))
 - Lower default max compressed replay recording segment size to 10 MiB. ([#2031](https://github.com/getsentry/relay/pull/2031))
+- Increase chunking limit to 15MB for replay recordings. ([#2032](https://github.com/getsentry/relay/pull/2032))
 - Add a data category for indexed profiles. ([#2051](https://github.com/getsentry/relay/pull/2051))
 
 ## 23.4.0

relay-config/src/config.rs

@@ -547,6 +547,8 @@ struct Limits {
     /// The maximum payload size for an uncompressed replay.
     #[serde(alias = "max_replay_size")]
     max_replay_uncompressed_size: ByteSize,
+    /// The maximum size for a replay recording Kafka message.
+    max_replay_message_size: ByteSize,
     /// The maximum number of threads to spawn for CPU and web work, each.
     ///
     /// The total number of threads spawned will roughly be `2 * max_thread_count + 1`. Defaults to
@@ -582,6 +584,7 @@ impl Default for Limits {
             max_profile_size: ByteSize::mebibytes(50),
             max_replay_compressed_size: ByteSize::mebibytes(10),
             max_replay_uncompressed_size: ByteSize::mebibytes(100),
+            max_replay_message_size: ByteSize::mebibytes(15),
             max_thread_count: num_cpus::get(),
             query_timeout: 30,
             shutdown_timeout: 10,
@@ -1834,6 +1837,15 @@ impl Config {
         self.values.limits.max_replay_uncompressed_size.as_bytes()
     }
 
+    /// Returns the maximum message size for an uncompressed replay.
+    ///
+    /// This is greater than max_replay_compressed_size because
+    /// it can include additional metadata about the replay in
+    /// addition to the recording.
+    pub fn max_replay_message_size(&self) -> usize {
+        self.values.limits.max_replay_message_size.as_bytes()
+    }
+
     /// Returns the maximum number of active requests
     pub fn max_concurrent_requests(&self) -> usize {
         self.values.limits.max_concurrent_requests

relay-server/src/actors/store.rs

@@ -664,14 +664,11 @@ impl StoreService {
         // message because we can achieve better parallelism when dealing with a single
         // message.
 
-        // Max message size is 1MB.
-        let max_message_size = 1000 * 1000;
-
         // 2000 bytes are reserved for the message metadata.
         let max_message_metadata_size = 2000;
 
         // Remaining bytes can be filled by the payload.
-        let max_payload_size = max_message_size - max_message_metadata_size;
+        let max_payload_size = self.config.max_replay_message_size() - max_message_metadata_size;
 
         if item.payload().len() < max_payload_size {
             let message =