-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature Request]: Refresh side input from BigQuery #26196
Comments
A solution for this is coming in 2.47.0 (the in progress release). The solution for this is to use the Slowly Updating Side Inputs pattern , using PeriodicImpulse. After the release, the examples in the linked doc will be updated accordingly. |
I wonder if you got this working with As example code:
Following issue occurs: |
|
@liferoad thanks for the help here. I was first trying to fetch info using just plain
|
This is supported by #13170 with |
What would you like to happen?
When you join a PubSub stream with BigQuery using side input, the side input data is loaded once and stays for the lifetime of the dataflow job, however there is no way to refresh the cache. Add the ability to be able to refresh the cache
Issue Priority
Priority: 3 (nice-to-have improvement)
Issue Components
The text was updated successfully, but these errors were encountered: