Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using StreamCuts and using readers from multiple process in the same reader group. #411

Open
junglie85 opened this issue Mar 23, 2023 · 1 comment

Comments

@junglie85
Copy link

I'm trying to understand how the Rust client handles management of the last read position in a stream so that readers can continue from the last read position.

Suppose the following scenario:

  1. Client A writes data to s1.
  2. Client B creates reader group rg1 and reader r1.
  3. B reads from s1.
  4. B crashes.
  5. Client A continues to write data to s1.
  6. B restarts, creates reader group rg1 and reader r1.
  7. B starts reading from last read position in the stream.

If I understand correctly, I can start consuming a stream from a StreamCut, but what isn't clear to me from looking at the source code is how I would create a stream cut in the first place. Is there an API for this?

And if I do have a stream cut, is it my responsibility to persist it somehow or does Pravega take care of this for me?

It's also not clear to me how this relates to checkpointing?

If I wanted to add client C in another process/machine as another reader in reader group rg1, how can I do this? There isn't an API call for joining an existing reader group. Should I use the create_reader_group API call, possibly with an existing ReaderGroupConfig (which I assume it is my responsibility to serialise and transmit from B to C somehow) or is there a more correct approach to this?

Any help given in these areas is much appreciated.

@ShwethaSNayak
Copy link
Contributor

@junglie85, At present there is no API which gives Stream Cut information.
Refer PR - #465 for stream cut API.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants