[Merged by Bors] - improvement: add helper method to get utf8 from data

[sehz]

Make it easier to get the utf8 string from record data, especially for consumers. This helps unnecessary conversion to owned unless it is necessary. Replaced existing examples with improved patterns to reduce unnecessary allocations and establish best practices for string conversion.

Also, add get_key() and get_value() to ConsumerRecord, which should be used rather than the existing value(), which returns &[u8]. Minor clean-up to remove unnecessary abstractions

Before:

while let Some(Ok(record)) = stream.next().await {
        let string = String::from_utf8_lossy(record.value());
        println!("{string}");
 }

After:

while let Some(Ok(record)) = stream.next().await {
      println!("{}", record.get_value().as_utf8_lossy_string());
}

[sehz]

bors r+

[ajhunyady]

I ran into this issue and was looking for get_value(). We also need additional convenience methods for reading & writing.
This is very complex:

fn json_array_to_records(record: &Record) -> Result<Vec<(Option<RecordData>, RecordData)>> {
    let array = serde_json::from_slice::<Vec<serde_json::Value>>(record.value.as_ref())?;

    // Convert each JSON value from the array back into a JSON string
    let strings: Vec<String> = array
        .into_iter()
        .map(|value| serde_json::to_string(&value))
        .collect::<core::result::Result<_, _>>()?;

    // Create one record from each JSON string to send
    let kvs: Vec<(Option<RecordData>, RecordData)> = strings
        .into_iter()
        .map(|s| (None, RecordData::from(s)))
        .collect();

    Ok(kvs)
}

[bors]

Pull request successfully merged into master.

Build succeeded!

The publicly hosted instance of bors-ng is deprecated and will go away soon.

If you want to self-host your own instance, instructions are here.
For more help, visit the forum.

If you want to switch to GitHub's built-in merge queue, visit their help page.

• Done

[sehz]

Create separate issue for that