AmqpConsumer::receiveBasicGet, only one message per timeout consumed #159
Labels
Comments
|
@dkarlovi Could you please provide more information about your setup and if possible code snippets. The libs versions, a framework if used, configs, do you use the enqueue client or amqp transport directly. do you ack messages? |
|
Sure, using:
Will try to downgrade to 0.5 to figure out where exactly the regression comes from. Might also disable my processors and just leave the built-in debug output. enqueue:
async_events:
enabled: true
# spool_producer: true
transport:
default: 'rabbitmq_amqp'
rabbitmq_amqp:
host: '%broker_host%'
port: '%broker_port%'
user: '%broker_user%'
pass: '%broker_password%'
vhost: '%broker_name%'
persisted: true
delay_plugin_installed: true
client:
traceable_producer: '%kernel.debug%'
job: true
extensions:
doctrine_ping_connection_extension: true
doctrine_clear_identity_map_extension: true
signal_extension: true |
|
It seems it's trying to read from the wrong queue and will try the proper queue once and then only the other queue until the timeout. If I echo the queue name attempted, it will look like When I set |
|
well that's expected behavior. the consume command by default subscribes to all known queues. It round-robins them all the time. Solutions:
|
|
@makasim but it only attempts to read from the default queue every 5sec while reading from the events queue every while loop (100msec). |
No, it is not. according to what you previously posted it consumes from two queues: "enqueue.app.default" and "symfony_events" |
|
Here's the output with timestamps: |
|
BTW yes, this differs from the previous behaviour, since I've added the timestamp, it will spend 5sec on each queue, not only on events. Don't know why, this is really weird. |
|
Well this actually makes sense, now that I think about it and read the code:
If you only have one consume command instance running, it will work exactly like this, it will NOT read from any queue ASAP but in You're right, it does work as designed, but I don't think it's something the user would expect here. |
|
Also figured out why I got only one read from default queue before: it was full. :) So, if you have two queues, Q1 and Q2, say Q1 is FULL of messages, Q2 is empty, you'll get You're spending most of your time monitoring the empty queue, not the full one. |
|
That's not good. |
|
You can say that again. :) Maybe rethink the timeout logic and go for round robin approach? Shorted the timeout significantly? |
makasim
changed the title
|
I am going to do next:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I've upgraded to 0.6 couple of days ago and I'm seeing a regression in
AmqpConsumer::receiveBasicGetbasically, it will only receive one message per timeout, for some reason.
Say I have say 10 messages in a queue, it will only receive the first one and stall, reach timeout, retry and then receive another. I've added some debug statements in the
while()loop and afterusleep(), it will go like:No matter how many messages there are ready to process in the queue. If I set
$timeout = 1to the top of the method, everything works as expected so it shouldn't be related to my processors being slow or hanging.I've cleared my RabbitMQ Docker volume to avoid any possible misconfiguration between 0.5 and 0.6, it did not help. Using ext/amqp 1.9.0 (which I also used on 0.5).
The text was updated successfully, but these errors were encountered: