Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support incrementally feeding into Conduits #507

Open
alexbiehl opened this issue Jun 19, 2023 · 1 comment
Open

Support incrementally feeding into Conduits #507

alexbiehl opened this issue Jun 19, 2023 · 1 comment

Comments

@alexbiehl
Copy link

alexbiehl commented Jun 19, 2023

At work we had a situation where we have a fairly complex sink implemented using Conduits:

uploadLogs :: ConduitT LogEntry output m ()

It so happened that the code that used uploadLogs required refactoring and we ended up having to implement a function

type Logger m = LogEntry -> m ()

to upload the logs. Of course we can't just reuse the uploadLogs conduit -- it handles chunking, compression, buffering, uploading. It would be quite handy and made Conduits more easily integrate in existing systems if there was a function

withIncrementalSink :: MonadUnliftIO m => Conduit input Void  m () -> ( (input -> m ()) -> m r ) -> m r
withIncrementalSink sink action = ... 

this allows action to feed inputs incrementally and once action returns we'd signal end-of-input to the sink and shut it down.

What do you think? This proposal clearly is not exactly Conduits main purpose but would pose an adapter that allows integrating Conduits more easily into all sorts of places in a modern Haskell codebase.

@snoyberg
Copy link
Owner

I don't see a problem with adding such a helper function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants