-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pub / Sub Publisher: Async publish breaks around 1000 messages #4575
Comments
@anorth2 I'm fairly certain it's a bug. I ran into it yesterday when reproducing a subscriber issue that required a large backlog of published messages. I got as high as 1375 messages through before failure, but not consistently. Thanks for filing. |
@dhermes Also wanted to ask, are there expected side effects from calling publisher.publish from a thread? if I use the following code |
@anorth2 To provide a quick answer: I don't know. Since I've started the "hardening" process on the highly concurrent Pub / Sub implementation (i.e. |
I have a reproducible sample that I am debugging here: https://github.com/dhermes/google-cloud-pubsub-performance/tree/master/publish-many |
Unit tests will be coming in the next commit. Fixes googleapis#4575.
Unit tests will be coming in the next commit. Fixes googleapis#4575.
danny, any idea when this will land on pypi?
|
Soon: see #4619 |
This is still not working for me, but with a new error: import time
from google.cloud import pubsub_v1
publisher = pubsub_v1.PublisherClient(batch_settings)
topic_path = publisher.topic_path('XXXX', 'XXX')
for n in range(1000):
data = u'Message number {}'.format(n)
def dt(n, t):
def g(message_id):
print "%g\t%g\t%g"%(n, time.clock() - t , t)
return g
data = data.encode('utf-8')
publisher.publish(topic_path, data=data).add_done_callback(dt(n, time.clock()))
Errors (many of these:)
|
Discussed with @kir-titievsky on Hangouts. Seems to be an intermittent HTTP issue (the error indicates that |
Am unsure if this is a bug (can't find documentation explaining this behavior).
I can send up to 999 pubsub messages and wait for results before continuing, but if I try to send 1000, none of them complete. However, if I run in debug mode, they will complete (because I have a breakpoint in the code allowing more execution time).
It exhibits this failure both in the main thread, or calling Publisher.publish from a thread.
OS type and version
MacOS High Sierra
Python version and virtual environment information
python --version
Python 3.6.3
google-cloud-python version
pip show google-cloud
,pip show google-<service>
orpip freeze
google-api-core==0.1.2
google-auth==1.2.1
google-cloud-core==0.28.0
google-cloud-pubsub==0.29.2
google-gax==0.15.16
googleapis-common-protos==1.5.3
grpc-google-iam-v1==0.11.4
None, hanging, no exception.
Publish 1000 messages storing their futures. None of the futures will complete.
Goal here is to send specified number of messages per second and sleep.
The text was updated successfully, but these errors were encountered: