Add Source <> InputStream operations and chunks

[kciesielski]

No description provided.

[adamw]

NonFatal! :)

[adamw]

we need a linter for that ;)

[adamw]

ah no sorry 🤦 stream operators catch IE on purpose ... as these are propagated downstream. I discover this not the first time, I just need to document it somewhere visible ... ;)

[kciesielski]

This, and also I copied that from your code :) softwaremill/sttp#2158

[adamw]

hehe yes, well, it just shows it's not that intuitive ;)

[adamw]

it does? it looks like a wrapper on an array, are there any optimizations in palce?

[kciesielski]

There are no optimizations yet, that's why I wrote that it "should" use these optimizations. I intended this PR to expose the Chunk API but start with simple non-optimized array operations. It can then later evolved as needed.

[adamw]

Ok, sounds good, let's just explicitly write that: Chunk for now is a think wrapper on arrays, in the future we want to optimize its operations

[adamw]

Apart from chunk - looks good :) I mean, chunk is ok as well, but I'm not sure if it's complete

[adamw]

Thanks :) Can you add follow-up tasks to (a) optimize the chunks, (b) add more input/output methods, I think reading/writing to/from File/Path would be a good start, anything else that you think is essential? Maybe also check with fs2/akka streams for inspiration

[kciesielski]

Chunk task: #136
As for other operations I'd consider essential:

• It may be useful to have a _.chunk method for Source[Byte], especially if we want to represent byte streams like this in Tapir. Conversely - _.unchunk.
• I/O: File read/write, maybe also Sink.fromOutputStream?
• Text: we could consider adding methods to decode/encode between Source[Chunk[Byte]] and Source[String] with given charset (UTF-8 for a start).
• Reactive streams integration

[adamw]

I/O: File read/write, maybe also Sink.fromOutputStream?

Yes, or just source.toOutputStream(OutputStream), same as you might have source.toFile(File) (or Path). We could also write a Source[String] to a file, just as a Source[Chunk[Byte]].

Text: we could consider adding methods to decode/encode between Source[Chunk[Byte]] and Source[String] with given charset (UTF-8 for a start).

I guess .lines is such a method?

But let's open the tasks, we can discuss there further.

[kciesielski]

Ok, tasks created:
#138
#137

I think encode/decode will be a different thing than .lines, not assuming line breaks but working on all kinds of text streams continually.

[adamw]

Thanks :) Ok, we'll need to document this properly then.