php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Request #59965 Support of pre-fetch.
Submitted: 2011-09-23 14:39 UTC Modified: 2011-10-26 02:02 UTC
From: andrew dot hobson at moveonlineretail dot com Assigned: pdezwart (profile)
Status: Closed Package: amqp (PECL)
PHP Version: 5.2.17 OS: Linux 2.6.18-238.12.1.el5.028st
Private report: No CVE-ID: None
View Add Comment Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
You can add a comment by following this link or if you reported this bug, you can edit this bug over here.
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: andrew dot hobson at moveonlineretail dot com
New email:
PHP Version: OS:

 

 [2011-09-23 14:39 UTC] andrew dot hobson at moveonlineretail dot com
Description:
------------
After discussing this on the rabbitmq-discuss forum it was 
suggested that my problem may be being caused by my call to 
consume() actually fetching all messages in the queue, 
rather than just the top one. I am setting min to 0 and max 
to 1 in the hope that this would fetch only one message.

The scenario is that I have one reader process which may die 
due to system failure and at restart I'd like the same 
single consumer process to start up again and pick up where 
it left off. So a summary of what I'd like to happen is:

(1) My transmitter process sends and queues up messages 1, 
2, 3, 4, 5 & 6.
(2) My single consumer process reads and acknowledges 
messages 1 and 2 in sequence.
(3) My consumer process reads message 3 and then dies 
without acknowledging.
(4) I restart my consumer process which will request the 
next message, which should be the unacknowledged message 3.
(5) My consumer process would then continue reading messages 
from the queue (4, 5, & 6) in sequence.

What I am experiencing is that when my consumer process is 
restarted messages 3, 4, 5 & 6 are all delivered out of 
sequence and all with a redelivery count of 1. I understand 
that message 3 might be out of sequence because it was 
unacknowledged and from what I have read about RabbitMQ 
message handling, when a message is redelivered then it's 
sequence is no longer guaranteed.

However I'm not clear as to why the remaining queued 
messages are out of sequence too. This is where the 
explanation of my code possibly fetching all messages in the 
queue (1 to 6) in the call to consume() and when it dies, 
causing RabbitMQ to redelivery all of them, resulting in 
them being out of sequence.

So the 'bug', if you can call it that, is to ask whether it 
is possible to set a pre-fetch value which only gets a 
specified number of messages from the queue, which in this 
scenario I would want to set to 1. That way hopefully only 
message 3 would get redelivered by RabbitMQ because it was 
not acknowledged, and the other messages would still be 
delivered in sequence because they haven't yet been read 
from the queue.

Hope this makes sense.



Reproduce code:
---------------
#!/usr/bin/php -q
<?php

// config
$exchangeName = 'HERMES_EXCHANGE';
$routingKey = 'DIRECT_ROUTE';
$queueName = 'HERMES_QUEUE';

// connect
$connection = new AMQPConnection();
$connection->connect();

// setup our queue
$q = new AMQPQueue($connection);
// Create the queue to be:
//      AMQP_DURABLE - messages will withstand a broker restart (i.e. they are written to disk).
//      AMQP_NOACK - when consumed messages will not be marked as delivered until an explicit ACK is received.
$q->declare($queueName, AMQP_DURABLE | AMQP_NOACK );

// Bind it on the exchange to routing key.
$q->bind($exchangeName, $routingKey);

// Set the options for our consumption of the messages:
//  Get a minimum of 0 msg.
//  Get a maximum of 1 msg.
//  Don't ACK the message on consumption i.e. explicitly acknowledge later.
$options = array(
 'min' =0,
 'max' =1,
 'ack' =false
);

$polling_count = 0;
while (true) {
        $polling_count++;
        echo "\r\nPolling. Count: " . $polling_count . "\r\n";

        // Get the messages
        $results_array = $q->consume($options);

        // show the message
        $delivery_tag =  $results_array[0]['delivery_tag'];
        $redelivered =  $results_array[0]['redelivered'];
        $message_body =  $results_array[0]['message_body'];

        echo "Message read.\r\n";
        echo 'delivery_tag: [' . $delivery_tag . "].\r\n";
        echo 'redelivered: [' . $redelivered . "].\r\n";
        echo 'message_body: [' . $message_body . "].\r\n";

        if ($polling_count != 3) {
                if ($delivery_tag != 0) {
                        // Acknowledge receipt of the message if we had a valid delivery tag.
                        $result = $q->ack($delivery_tag);
                        echo 'Acknowledge result: ' . $result . "\r\n";
                }
        } else {
                echo 'No Ack';
                
        }

        usleep(4000000);
}

// Disconnect
$connection->disconnect();

?>

Expected result:
----------------
In the code above I pre-populated the message queue with 6 
messages from a separate transmit script, then ran this 
consuming process. After the third message was received I 
interrupted the program with ctrl-c and then re-ran it.

I expected that the sequence of messages read after the 
interrupt would be 4, 5, & 6 and then 3, due to the re-queuing 
of 3 following failure by my script to acknowledge message 3.

Actual result:
--------------
After interrupting the program and re-reading from the queue 
the sequence of messages 3 - 6 is completely unpredictable.

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2011-10-26 02:02 UTC] pdezwart@php.net
-Status: Open +Status: Closed -Assigned To: +Assigned To: pdezwart
 [2011-10-26 02:02 UTC] pdezwart@php.net
This bug has been fixed in SVN.

Snapshots of the sources are packaged every three hours; this change
will be in the next snapshot. You can grab the snapshot at
http://snaps.php.net/.

 For Windows:

http://windows.php.net/snapshots/
 
Thank you for the report, and for helping us make PHP better.

This is fixed in trunk and will be part of the v1.0 release, coming soon.
 
PHP Copyright © 2001-2021 The PHP Group
All rights reserved.
Last updated: Sun Jul 25 15:01:23 2021 UTC