php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #59198 multiple srv. + ketama + cas = infinite loop
Submitted: 2010-05-08 06:59 UTC Modified: 2012-03-07 17:30 UTC
From: abodera at gmail dot com Assigned: andrei (profile)
Status: Closed Package: memcached (PECL)
PHP Version: 5.3.2 OS: Fedora 12
Private report: No CVE-ID: None
View Add Comment Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
You can add a comment by following this link or if you reported this bug, you can edit this bug over here.
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: abodera at gmail dot com
New email:
PHP Version: OS:

 

 [2010-05-08 06:59 UTC] abodera at gmail dot com
Description:
------------
That's odd. It happens with latest MASTER branch and SVN short after  1.0.1. I have reverted to 1.0.1 to work around it. Freshly tested on SVN + libmemcached 0.40

Scenario: 2 servers (1.4.4), ketama ON, distribution DEF/CONSISTENT (irrelevant), binary ON/OFF (irrelevant), persistent ON.

Operations performed (1st run):
1. fetch KEY -> result: 16 - NOT FOUND
2. add KEY -> result: 0 - SUCCESS
3. [END]

Operations performed (2nd run)
1. fetch KEY -> result: 0 - SUCCESS (found, cas=X)
2. CAS(X,KEY,val) -> result: 16 - NOT FOUND

 ??? confusion, why is it not found ???

3. loop again
4. fetch KEY -> result: 0 - SUCCESS (found, cas=X)

 ??? same CAS, so the value has not changed! ???

5. CAS(X,KEY,val) -> result: 16 - NOT FOUND

 ??? why is it missing ???

6. ..... infinite loop .......


WORKAROUND1: Roll back to 1.0.1 - works flawlessly
WORKAROUND2: Disable KETAMA (?) Seems to work without it.


Below is a code taken from manual, with some added verbosity and a loop iteration limit ($x<2 for debugging).


Reproduce code:
---------------
<?php
$m = new Memcached('slamtest1');
if(!count($m->getServerList())){
	$m->addServers(array(
		array('127.0.0.101',11211),
		array('127.0.0.102',11211),
	));
	$m->setOption(Memcached::OPT_LIBKETAMA_COMPATIBLE, true);
	$m->setOption(Memcached::OPT_DISTRIBUTION,Memcached::DISTRIBUTION_CONSISTENT);
}

echo "Opt: Ketama: ";
var_dump($m->getOption(Memcached::OPT_LIBKETAMA_COMPATIBLE));
echo "Opt: distribution: ";
var_dump($m->getOption(Memcached::OPT_DISTRIBUTION));
echo "Opt: binary protocol: ";
var_dump($m->getOption(Memcached::OPT_BINARY_PROTOCOL));

print_r($m->getServerList());
print_r($m->getVersion());

set_time_limit(5);error_reporting(E_ALL);ini_set('display_errors',1);
ini_set('output_buffering',0);
ob_end_flush();

// -- do a increase test
$key = 'slamtest_1_incr';
$x = 0;
do{
	$x++;
	// -- fetch the value
	$cas = 0.0;
	$success = false;
	echo "Fetching $key\n";flush();
	$val = $m->get($key,null,$cas);
	echo "after fetch, RESULT: ".$m->getResultCode()." - ".$m->getResultMessage()."\n";flush();
	echo "VAL: $val\n";
	if ($m->getResultCode() == Memcached::RES_NOTFOUND) {
		echo "Not found - adding\n";
		// -- value does not exist yet
		if($m->add($key, 1)){
			echo "add success! RESULT: ".$m->getResultCode()." - ".$m->getResultMessage()."\n";flush();
			$success = 1;
		}else{
			echo "add failed! RESULT: ".$m->getResultCode()." - ".$m->getResultMessage()."\n";flush();
		}
	}else{
		echo "Found!\n";flush();
		echo "CAS: (".gettype($cas).") $cas\n";flush();
		// -- increase the value by 1
		$val++;

		// -- store it
		if($m->cas($cas,$key,$val)){
			echo "CAS success\n";flush();
			$success = true;
		}else{
			echo "CAS failed, because ".$m->getResultCode()." - ".$m->getResultMessage()."\n";flush();
		}
	}
	echo "LOOP RESULT: ".$m->getResultCode()." - ".$m->getResultMessage()."\n";;flush();
	if(!$success) usleep(mt_rand(10000,100000));
}while($x<2 && !$success);

echo "Finished after $x iters!\n";exit();


Expected result:
----------------
#### THE VERY SAME CODE RUN ON VER 1.0.1 AND LIBMEMCACHED 0.38 WITH THE SAME PAIR OF MEMCACHED DAEMONS ####
Opt: Ketama: int(1)
Opt: distribution: int(1)
Opt: binary protocol: int(0)
Array
(
    [0] => Array
        (
            [host] => 127.0.0.101
            [port] => 11211
            [weight] => 1
        )

    [1] => Array
        (
            [host] => 127.0.0.102
            [port] => 11211
            [weight] => 1
        )

)
Array
(
    [127.0.0.101:11211] => 1.4.4
    [127.0.0.102:11211] => 1.4.4
)
Fetching slamtest_1_incr
after fetch, RESULT: 0 - SUCCESS
VAL: 3
Found!
CAS: (double) 730849
CAS success
LOOP RESULT: 0 - SUCCESS
Finished (1)!


Actual result:
--------------
#### CODE RUN ON TRUNK + LIBMEMCACHED 0.40 WITH THE SAME PAIR OF MEMCACHED DAEMONS ####
Opt: Ketama: int(1)
Opt: distribution: int(1)
Opt: binary protocol: int(0)
Array
(
    [0] => Array
        (
            [host] => 127.0.0.101
            [port] => 11211
            [weight] => 1
        )

    [1] => Array
        (
            [host] => 127.0.0.102
            [port] => 11211
            [weight] => 1
        )

)
Array
(
    [127.0.0.101:11211] => 1.4.4
    [127.0.0.102:11211] => 1.4.4
)
Fetching slamtest_1_incr
after fetch, RESULT: 0 - SUCCESS
VAL: 3
Found!
CAS: (double) 641304
CAS failed, because 16 - NOT FOUND
LOOP RESULT: 16 - NOT FOUND
Fetching slamtest_1_incr
after fetch, RESULT: 0 - SUCCESS
VAL: 3
Found!
CAS: (double) 641304
CAS failed, because 16 - NOT FOUND
LOOP RESULT: 16 - NOT FOUND
Finished (2)!

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2010-05-08 07:00 UTC] abodera at gmail dot com
(typo in summary, sorry)
 [2010-05-08 07:15 UTC] abodera at gmail dot com
Simpler test case, CLI friendly. You can pass an argument which modifies the memcache key that will be used, thus storing it on another server. i.e.
> php test.php 4

After second run it will display "Infinite loop detected..."

<?php
$prefix = isset($argv[1]) ? (int)$argv[1] : 1;
$m = new Memcached('casmiss');
if(!count($m->getServerList())){
    $m->addServers(array(
        array('127.0.0.101',11211),
        array('127.0.0.102',11211),
    ));
    $m->setOption(Memcached::OPT_LIBKETAMA_COMPATIBLE, true);
    $m->setOption(Memcached::OPT_DISTRIBUTION,Memcached::DISTRIBUTION_CONSISTENT);
}
$key = 'casmisstest_'.$prefix.'_incr'; $x = 0;
$x = 0;$maxIter = 100;

do{
    $x++;
    $cas = 0.0;
    $success = false;
    $val = $m->get($key,null,$cas);
    if ($m->getResultCode() == Memcached::RES_NOTFOUND) {
        if($m->add($key, 1)){
            $success = 1;
        }else{
        }
    }else{
        $val++;
        $success = $m->cas($cas,$key,$val);
    }
}while($x<$maxIter && !$success);

if($x == $maxIter){
    echo "Infinite loop detected - $x iterations.";
    echo "Last result: ".$m->getResultMessage()."\n";
}else{
    echo "Finished after $x iters!\n";
}
 [2010-05-08 14:13 UTC] abodera at gmail dot com
Tested on 1.0.2 + libmemcached 0.40 - works OK
 [2012-03-07 17:30 UTC] andrei@php.net
-Status: Open +Status: Closed -Assigned To: +Assigned To: andrei
 [2012-03-07 17:30 UTC] andrei@php.net
Closing because of age. Please try version 2.0.1 and reopen if this still occurs.
 
PHP Copyright © 2001-2020 The PHP Group
All rights reserved.
Last updated: Fri Dec 04 18:01:23 2020 UTC