php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Request #59017 ketama hashing support added
Submitted: 2009-12-31 05:53 UTC Modified: 2013-02-18 00:35 UTC
From: marlon at hyves dot nl Assigned:
Status: No Feedback Package: memcache (PECL)
PHP Version: 5.2.10 OS:
Private report: No CVE-ID: None
Have you experienced this issue?
Rate the importance of this bug to you:

 [2009-12-31 05:53 UTC] marlon at hyves dot nl
Description:
------------
I've ported the relevant code from libketama into pecl-memcache-3.0.4, tested and is 100% compatible.

Actual result:
--------------
diff -ruN memcache-3.0.4/config9.m4 memcache-3.0.4-ketama/config9.m4
--- memcache-3.0.4/config9.m4	2009-02-22 17:01:43.000000000 +0100
+++ memcache-3.0.4-ketama/config9.m4	2009-12-31 11:26:19.000000000 +0100
@@ -92,7 +92,7 @@
     AC_MSG_RESULT([enabled])
     AC_DEFINE(HAVE_MEMCACHE_SESSION,1,[Whether memcache session handler is enabled])
     AC_DEFINE(HAVE_MEMCACHE,1,[Whether you want memcache support])
-    PHP_NEW_EXTENSION(memcache, memcache.c memcache_pool.c memcache_queue.c memcache_ascii_protocol.c memcache_binary_protocol.c memcache_standard_hash.c memcache_consistent_hash.c memcache_session.c, $ext_shared,,-I$session_inc_path)
+    PHP_NEW_EXTENSION(memcache, memcache.c memcache_pool.c memcache_queue.c memcache_ascii_protocol.c memcache_binary_protocol.c memcache_standard_hash.c memcache_consistent_hash.c memcache_ketama_hash.c md5.c memcache_session.c, $ext_shared,,-I$session_inc_path)
     ifdef([PHP_ADD_EXTENSION_DEP],
     [
       PHP_ADD_EXTENSION_DEP(memcache, session)
@@ -100,7 +100,7 @@
   else 
     AC_MSG_RESULT([disabled])
     AC_DEFINE(HAVE_MEMCACHE,1,[Whether you want memcache support])
-    PHP_NEW_EXTENSION(memcache, memcache.c memcache_pool.c memcache_queue.c memcache_ascii_protocol.c memcache_binary_protocol.c memcache_standard_hash.c memcache_consistent_hash.c, $ext_shared)
+    PHP_NEW_EXTENSION(memcache, memcache.c memcache_pool.c memcache_queue.c memcache_ascii_protocol.c memcache_binary_protocol.c memcache_standard_hash.c memcache_consistent_hash.c memcache_ketama_hash.c md5.c, $ext_shared)
   fi
 
 dnl this is needed to build the extension with phpize and -Wall
@@ -109,4 +109,5 @@
     CFLAGS="$CFLAGS -Wall"
   fi
 
+  LDFLAGS="$LDFLAGS -lm"
 fi
diff -ruN memcache-3.0.4/md5.c memcache-3.0.4-ketama/md5.c
--- memcache-3.0.4/md5.c	1970-01-01 01:00:00.000000000 +0100
+++ memcache-3.0.4-ketama/md5.c	2009-12-31 11:26:04.000000000 +0100
@@ -0,0 +1,403 @@
+/*
+  Copyright (C) 1999, 2000, 2002 Aladdin Enterprises.  All rights reserved.
+
+  This software is provided 'as-is', without any express or implied
+  warranty.  In no event will the authors be held liable for any damages
+  arising from the use of this software.
+
+  Permission is granted to anyone to use this software for any purpose,
+  including commercial applications, and to alter it and redistribute it
+  freely, subject to the following restrictions:
+
+  1. The origin of this software must not be misrepresented; you must not
+     claim that you wrote the original software. If you use this software
+     in a product, an acknowledgment in the product documentation would be
+     appreciated but is not required.
+  2. Altered source versions must be plainly marked as such, and must not be
+     misrepresented as being the original software.
+  3. This notice may not be removed or altered from any source distribution.
+
+  L. Peter Deutsch
+  ghost@aladdin.com
+
+ */
+/* $Id: md5.c,v 1.6 2002/04/13 19:20:28 lpd Exp $ */
+/*
+  Independent implementation of MD5 (RFC 1321).
+
+  This code implements the MD5 Algorithm defined in RFC 1321, whose
+  text is available at
+	http://www.ietf.org/rfc/rfc1321.txt
+  The code is derived from the text of the RFC, including the test suite
+  (section A.5) but excluding the rest of Appendix A.  It does not include
+  any code or documentation that is identified in the RFC as being
+  copyrighted.
+
+  The original and principal author of md5.c is L. Peter Deutsch
+  <ghost@aladdin.com>.  Other authors are noted in the change history
+  that follows (in reverse chronological order):
+
+  2002-04-13 lpd Clarified derivation from RFC 1321; now handles byte order
+	either statically or dynamically; added missing #include <string.h>
+	in library.
+  2002-03-11 lpd Corrected argument list for main(), and added int return
+	type, in test program and T value program.
+  2002-02-21 lpd Added missing #include <stdio.h> in test program.
+  2000-07-03 lpd Patched to eliminate warnings about "constant is
+	unsigned in ANSI C, signed in traditional"; made test program
+	self-checking.
+  1999-11-04 lpd Edited comments slightly for automatic TOC extraction.
+  1999-10-18 lpd Fixed typo in header comment (ansi2knr rather than md5).
+  1999-05-03 lpd Original version.
+ */
+
+#include "md5.h"
+#include <string.h>
+
+#undef BYTE_ORDER	/* 1 = big-endian, -1 = little-endian, 0 = unknown */
+#ifdef ARCH_IS_BIG_ENDIAN
+#  define BYTE_ORDER (ARCH_IS_BIG_ENDIAN ? 1 : -1)
+#else
+#  define BYTE_ORDER 0
+#endif
+
+#define T_MASK ((md5_word_t)~0)
+#define T1 /* 0xd76aa478 */ (T_MASK ^ 0x28955b87)
+#define T2 /* 0xe8c7b756 */ (T_MASK ^ 0x173848a9)
+#define T3    0x242070db
+#define T4 /* 0xc1bdceee */ (T_MASK ^ 0x3e423111)
+#define T5 /* 0xf57c0faf */ (T_MASK ^ 0x0a83f050)
+#define T6    0x4787c62a
+#define T7 /* 0xa8304613 */ (T_MASK ^ 0x57cfb9ec)
+#define T8 /* 0xfd469501 */ (T_MASK ^ 0x02b96afe)
+#define T9    0x698098d8
+#define T10 /* 0x8b44f7af */ (T_MASK ^ 0x74bb0850)
+#define T11 /* 0xffff5bb1 */ (T_MASK ^ 0x0000a44e)
+#define T12 /* 0x895cd7be */ (T_MASK ^ 0x76a32841)
+#define T13    0x6b901122
+#define T14 /* 0xfd987193 */ (T_MASK ^ 0x02678e6c)
+#define T15 /* 0xa679438e */ (T_MASK ^ 0x5986bc71)
+#define T16    0x49b40821
+#define T17 /* 0xf61e2562 */ (T_MASK ^ 0x09e1da9d)
+#define T18 /* 0xc040b340 */ (T_MASK ^ 0x3fbf4cbf)
+#define T19    0x265e5a51
+#define T20 /* 0xe9b6c7aa */ (T_MASK ^ 0x16493855)
+#define T21 /* 0xd62f105d */ (T_MASK ^ 0x29d0efa2)
+#define T22    0x02441453
+#define T23 /* 0xd8a1e681 */ (T_MASK ^ 0x275e197e)
+#define T24 /* 0xe7d3fbc8 */ (T_MASK ^ 0x182c0437)
+#define T25    0x21e1cde6
+#define T26 /* 0xc33707d6 */ (T_MASK ^ 0x3cc8f829)
+#define T27 /* 0xf4d50d87 */ (T_MASK ^ 0x0b2af278)
+#define T28    0x455a14ed
+#define T29 /* 0xa9e3e905 */ (T_MASK ^ 0x561c16fa)
+#define T30 /* 0xfcefa3f8 */ (T_MASK ^ 0x03105c07)
+#define T31    0x676f02d9
+#define T32 /* 0x8d2a4c8a */ (T_MASK ^ 0x72d5b375)
+#define T33 /* 0xfffa3942 */ (T_MASK ^ 0x0005c6bd)
+#define T34 /* 0x8771f681 */ (T_MASK ^ 0x788e097e)
+#define T35    0x6d9d6122
+#define T36 /* 0xfde5380c */ (T_MASK ^ 0x021ac7f3)
+#define T37 /* 0xa4beea44 */ (T_MASK ^ 0x5b4115bb)
+#define T38    0x4bdecfa9
+#define T39 /* 0xf6bb4b60 */ (T_MASK ^ 0x0944b49f)
+#define T40 /* 0xbebfbc70 */ (T_MASK ^ 0x4140438f)
+#define T41    0x289b7ec6
+#define T42 /* 0xeaa127fa */ (T_MASK ^ 0x155ed805)
+#define T43 /* 0xd4ef3085 */ (T_MASK ^ 0x2b10cf7a)
+#define T44    0x04881d05
+#define T45 /* 0xd9d4d039 */ (T_MASK ^ 0x262b2fc6)
+#define T46 /* 0xe6db99e5 */ (T_MASK ^ 0x1924661a)
+#define T47    0x1fa27cf8
+#define T48 /* 0xc4ac5665 */ (T_MASK ^ 0x3b53a99a)
+#define T49 /* 0xf4292244 */ (T_MASK ^ 0x0bd6ddbb)
+#define T50    0x432aff97
+#define T51 /* 0xab9423a7 */ (T_MASK ^ 0x546bdc58)
+#define T52 /* 0xfc93a039 */ (T_MASK ^ 0x036c5fc6)
+#define T53    0x655b59c3
+#define T54 /* 0x8f0ccc92 */ (T_MASK ^ 0x70f3336d)
+#define T55 /* 0xffeff47d */ (T_MASK ^ 0x00100b82)
+#define T56 /* 0x85845dd1 */ (T_MASK ^ 0x7a7ba22e)
+#define T57    0x6fa87e4f
+#define T58 /* 0xfe2ce6e0 */ (T_MASK ^ 0x01d3191f)
+#define T59 /* 0xa3014314 */ (T_MASK ^ 0x5cfebceb)
+#define T60    0x4e0811a1
+#define T61 /* 0xf7537e82 */ (T_MASK ^ 0x08ac817d)
+#define T62 /* 0xbd3af235 */ (T_MASK ^ 0x42c50dca)
+#define T63    0x2ad7d2bb
+#define T64 /* 0xeb86d391 */ (T_MASK ^ 0x14792c6e)
+
+
+static void
+md5_process(md5_state_t *pms, const md5_byte_t *data /*[64]*/)
+{
+    md5_word_t
+	a = pms->abcd[0], b = pms->abcd[1],
+	c = pms->abcd[2], d = pms->abcd[3];
+    md5_word_t t;
+#if BYTE_ORDER > 0
+    /* Define storage only for big-endian CPUs. */
+    md5_word_t X[16];
+#else
+    /* Define storage for little-endian or both types of CPUs. */
+    md5_word_t xbuf[16];
+    const md5_word_t *X;
+#endif
+
+    {
+#if BYTE_ORDER == 0
+	/*
+	 * Determine dynamically whether this is a big-endian or
+	 * little-endian machine, since we can use a more efficient
+	 * algorithm on the latter.
+	 */
+	static const int w = 1;
+
+	if (*((const md5_byte_t *)&w)) /* dynamic little-endian */
+#endif
+#if BYTE_ORDER <= 0		/* little-endian */
+	{
+	    /*
+	     * On little-endian machines, we can process properly aligned
+	     * data without copying it.
+	     */
+	    if (!((data - (const md5_byte_t *)0) & 3)) {
+		/* data are properly aligned */
+		X = (const md5_word_t *)data;
+	    } else {
+		/* not aligned */
+		memcpy(xbuf, data, 64);
+		X = xbuf;
+	    }
+	}
+#endif
+#if BYTE_ORDER == 0
+	else			/* dynamic big-endian */
+#endif
+#if BYTE_ORDER >= 0		/* big-endian */
+	{
+	    /*
+	     * On big-endian machines, we must arrange the bytes in the
+	     * right order.
+	     */
+	    const md5_byte_t *xp = data;
+	    int i;
+
+#  if BYTE_ORDER == 0
+	    X = xbuf;		/* (dynamic only) */
+#  else
+#    define xbuf X		/* (static only) */
+#  endif
+	    for (i = 0; i < 16; ++i, xp += 4)
+		xbuf[i] = xp[0] + (xp[1] << 8) + (xp[2] << 16) + (xp[3] << 24);
+	}
+#endif
+    }
+
+#define ROTATE_LEFT(x, n) (((x) << (n)) | ((x) >> (32 - (n))))
+
+    /* Round 1. */
+    /* Let [abcd k s i] denote the operation
+       a = b + ((a + F(b,c,d) + X[k] + T[i]) <<< s). */
+#define F(x, y, z) (((x) & (y)) | (~(x) & (z)))
+#define SET(a, b, c, d, k, s, Ti)\
+  t = a + F(b,c,d) + X[k] + Ti;\
+  a = ROTATE_LEFT(t, s) + b
+    /* Do the following 16 operations. */
+    SET(a, b, c, d,  0,  7,  T1);
+    SET(d, a, b, c,  1, 12,  T2);
+    SET(c, d, a, b,  2, 17,  T3);
+    SET(b, c, d, a,  3, 22,  T4);
+    SET(a, b, c, d,  4,  7,  T5);
+    SET(d, a, b, c,  5, 12,  T6);
+    SET(c, d, a, b,  6, 17,  T7);
+    SET(b, c, d, a,  7, 22,  T8);
+    SET(a, b, c, d,  8,  7,  T9);
+    SET(d, a, b, c,  9, 12, T10);
+    SET(c, d, a, b, 10, 17, T11);
+    SET(b, c, d, a, 11, 22, T12);
+    SET(a, b, c, d, 12,  7, T13);
+    SET(d, a, b, c, 13, 12, T14);
+    SET(c, d, a, b, 14, 17, T15);
+    SET(b, c, d, a, 15, 22, T16);
+#undef SET
+
+     /* Round 2. */
+     /* Let [abcd k s i] denote the operation
+          a = b + ((a + G(b,c,d) + X[k] + T[i]) <<< s). */
+#define G(x, y, z) (((x) & (z)) | ((y) & ~(z)))
+#define SET(a, b, c, d, k, s, Ti)\
+  t = a + G(b,c,d) + X[k] + Ti;\
+  a = ROTATE_LEFT(t, s) + b
+     /* Do the following 16 operations. */
+    SET(a, b, c, d,  1,  5, T17);
+    SET(d, a, b, c,  6,  9, T18);
+    SET(c, d, a, b, 11, 14, T19);
+    SET(b, c, d, a,  0, 20, T20);
+    SET(a, b, c, d,  5,  5, T21);
+    SET(d, a, b, c, 10,  9, T22);
+    SET(c, d, a, b, 15, 14, T23);
+    SET(b, c, d, a,  4, 20, T24);
+    SET(a, b, c, d,  9,  5, T25);
+    SET(d, a, b, c, 14,  9, T26);
+    SET(c, d, a, b,  3, 14, T27);
+    SET(b, c, d, a,  8, 20, T28);
+    SET(a, b, c, d, 13,  5, T29);
+    SET(d, a, b, c,  2,  9, T30);
+    SET(c, d, a, b,  7, 14, T31);
+    SET(b, c, d, a, 12, 20, T32);
+#undef SET
+
+     /* Round 3. */
+     /* Let [abcd k s t] denote the operation
+          a = b + ((a + H(b,c,d) + X[k] + T[i]) <<< s). */
+#define H(x, y, z) ((x) ^ (y) ^ (z))
+#define SET(a, b, c, d, k, s, Ti)\
+  t = a + H(b,c,d) + X[k] + Ti;\
+  a = ROTATE_LEFT(t, s) + b
+     /* Do the following 16 operations. */
+    SET(a, b, c, d,  5,  4, T33);
+    SET(d, a, b, c,  8, 11, T34);
+    SET(c, d, a, b, 11, 16, T35);
+    SET(b, c, d, a, 14, 23, T36);
+    SET(a, b, c, d,  1,  4, T37);
+    SET(d, a, b, c,  4, 11, T38);
+    SET(c, d, a, b,  7, 16, T39);
+    SET(b, c, d, a, 10, 23, T40);
+    SET(a, b, c, d, 13,  4, T41);
+    SET(d, a, b, c,  0, 11, T42);
+    SET(c, d, a, b,  3, 16, T43);
+    SET(b, c, d, a,  6, 23, T44);
+    SET(a, b, c, d,  9,  4, T45);
+    SET(d, a, b, c, 12, 11, T46);
+    SET(c, d, a, b, 15, 16, T47);
+    SET(b, c, d, a,  2, 23, T48);
+#undef SET
+
+     /* Round 4. */
+     /* Let [abcd k s t] denote the operation
+          a = b + ((a + I(b,c,d) + X[k] + T[i]) <<< s). */
+#define I(x, y, z) ((y) ^ ((x) | ~(z)))
+#define SET(a, b, c, d, k, s, Ti)\
+  t = a + I(b,c,d) + X[k] + Ti;\
+  a = ROTATE_LEFT(t, s) + b
+     /* Do the following 16 operations. */
+    SET(a, b, c, d,  0,  6, T49);
+    SET(d, a, b, c,  7, 10, T50);
+    SET(c, d, a, b, 14, 15, T51);
+    SET(b, c, d, a,  5, 21, T52);
+    SET(a, b, c, d, 12,  6, T53);
+    SET(d, a, b, c,  3, 10, T54);
+    SET(c, d, a, b, 10, 15, T55);
+    SET(b, c, d, a,  1, 21, T56);
+    SET(a, b, c, d,  8,  6, T57);
+    SET(d, a, b, c, 15, 10, T58);
+    SET(c, d, a, b,  6, 15, T59);
+    SET(b, c, d, a, 13, 21, T60);
+    SET(a, b, c, d,  4,  6, T61);
+    SET(d, a, b, c, 11, 10, T62);
+    SET(c, d, a, b,  2, 15, T63);
+    SET(b, c, d, a,  9, 21, T64);
+#undef SET
+
+     /* Then perform the following additions. (That is increment each
+        of the four registers by the value it had before this block
+        was started.) */
+    pms->abcd[0] += a;
+    pms->abcd[1] += b;
+    pms->abcd[2] += c;
+    pms->abcd[3] += d;
+}
+
+void
+md5_init(md5_state_t *pms)
+{
+    pms->count[0] = pms->count[1] = 0;
+    pms->abcd[0] = 0x67452301;
+    pms->abcd[1] = /*0xefcdab89*/ T_MASK ^ 0x10325476;
+    pms->abcd[2] = /*0x98badcfe*/ T_MASK ^ 0x67452301;
+    pms->abcd[3] = 0x10325476;
+}
+
+void
+md5_append(md5_state_t *pms, const md5_byte_t *data, int nbytes)
+{
+    const md5_byte_t *p = data;
+    int left = nbytes;
+    int offset = (pms->count[0] >> 3) & 63;
+    md5_word_t nbits = (md5_word_t)(nbytes << 3);
+
+    if (nbytes <= 0)
+	return;
+
+    /* Update the message length. */
+    pms->count[1] += nbytes >> 29;
+    pms->count[0] += nbits;
+    if (pms->count[0] < nbits)
+	pms->count[1]++;
+
+    /* Process an initial partial block. */
+    if (offset) {
+	int copy = (offset + nbytes > 64 ? 64 - offset : nbytes);
+
+	memcpy(pms->buf + offset, p, copy);
+	if (offset + copy < 64)
+	    return;
+	p += copy;
+	left -= copy;
+	md5_process(pms, pms->buf);
+    }
+
+    /* Process full blocks. */
+    for (; left >= 64; p += 64, left -= 64)
+	md5_process(pms, p);
+
+    /* Process a final partial block. */
+    if (left)
+	memcpy(pms->buf, p, left);
+}
+
+void
+md5_finish(md5_state_t *pms, md5_byte_t digest[16])
+{
+    static const md5_byte_t pad[64] = {
+	0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+	0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
+    };
+    md5_byte_t data[8];
+    int i;
+
+    /* Save the length before padding. */
+    for (i = 0; i < 8; ++i)
+	data[i] = (md5_byte_t)(pms->count[i >> 2] >> ((i & 3) << 3));
+    /* Pad to 56 bytes mod 64. */
+    md5_append(pms, pad, ((55 - (pms->count[0] >> 3)) & 63) + 1);
+    /* Append the length. */
+    md5_append(pms, data, 8);
+    for (i = 0; i < 16; ++i)
+	digest[i] = (md5_byte_t)(pms->abcd[i >> 2] >> ((i & 3) << 3));
+}
+
+void md5_digest(const char* inString, unsigned char md5pword[16])
+{
+    md5_state_t md5state;
+
+	md5_init(&md5state);
+	md5_append(&md5state, (unsigned char *)inString, strlen(inString));
+	md5_finish(&md5state, md5pword);
+}
+
+unsigned int hash_key(const char* inString)
+{
+    unsigned char digest[16];
+
+    md5_digest(inString, digest);
+
+    return (unsigned int)(( digest[3] << 24 )
+                        | ( digest[2] << 16 )
+                        | ( digest[1] << 8 )
+                        | digest[0]);
+}
+
diff -ruN memcache-3.0.4/md5.h memcache-3.0.4-ketama/md5.h
--- memcache-3.0.4/md5.h	1970-01-01 01:00:00.000000000 +0100
+++ memcache-3.0.4-ketama/md5.h	2009-12-31 11:26:04.000000000 +0100
@@ -0,0 +1,102 @@
+/*
+  Copyright (C) 1999, 2002 Aladdin Enterprises.  All rights reserved.
+
+  This software is provided 'as-is', without any express or implied
+  warranty.  In no event will the authors be held liable for any damages
+  arising from the use of this software.
+
+  Permission is granted to anyone to use this software for any purpose,
+  including commercial applications, and to alter it and redistribute it
+  freely, subject to the following restrictions:
+
+  1. The origin of this software must not be misrepresented; you must not
+     claim that you wrote the original software. If you use this software
+     in a product, an acknowledgment in the product documentation would be
+     appreciated but is not required.
+  2. Altered source versions must be plainly marked as such, and must not be
+     misrepresented as being the original software.
+  3. This notice may not be removed or altered from any source distribution.
+
+  L. Peter Deutsch
+  ghost@aladdin.com
+
+ */
+/* $Id: md5.h,v 1.4 2002/04/13 19:20:28 lpd Exp $ */
+/*
+  Independent implementation of MD5 (RFC 1321).
+
+  This code implements the MD5 Algorithm defined in RFC 1321, whose
+  text is available at
+	http://www.ietf.org/rfc/rfc1321.txt
+  The code is derived from the text of the RFC, including the test suite
+  (section A.5) but excluding the rest of Appendix A.  It does not include
+  any code or documentation that is identified in the RFC as being
+  copyrighted.
+
+  The original and principal author of md5.h is L. Peter Deutsch
+  <ghost@aladdin.com>.  Other authors are noted in the change history
+  that follows (in reverse chronological order):
+
+  2002-04-13 lpd Removed support for non-ANSI compilers; removed
+	references to Ghostscript; clarified derivation from RFC 1321;
+	now handles byte order either statically or dynamically.
+  1999-11-04 lpd Edited comments slightly for automatic TOC extraction.
+  1999-10-18 lpd Fixed typo in header comment (ansi2knr rather than md5);
+	added conditionalization for C++ compilation from Martin
+	Purschke <purschke@bnl.gov>.
+  1999-05-03 lpd Original version.
+ */
+
+#ifndef md5_INCLUDED
+#  define md5_INCLUDED
+
+/*
+ * This package supports both compile-time and run-time determination of CPU
+ * byte order.  If ARCH_IS_BIG_ENDIAN is defined as 0, the code will be
+ * compiled to run only on little-endian CPUs; if ARCH_IS_BIG_ENDIAN is
+ * defined as non-zero, the code will be compiled to run only on big-endian
+ * CPUs; if ARCH_IS_BIG_ENDIAN is not defined, the code will be compiled to
+ * run on either big- or little-endian CPUs, but will run slightly less
+ * efficiently on either one than if ARCH_IS_BIG_ENDIAN is defined.
+ */
+
+typedef unsigned char md5_byte_t; /* 8-bit byte */
+typedef unsigned int md5_word_t; /* 32-bit word */
+
+/* Define the state of the MD5 Algorithm. */
+typedef struct md5_state_s {
+    md5_word_t count[2];	/* message length in bits, lsw first */
+    md5_word_t abcd[4];		/* digest buffer */
+    md5_byte_t buf[64];		/* accumulate block */
+} md5_state_t;
+
+#ifdef __cplusplus
+extern "C" 
+{
+#endif
+
+/* Initialize the algorithm. */
+
+#ifdef WIN32
+_declspec(dllexport)
+#endif
+void md5_init(md5_state_t *pms);
+
+/* Append a string to the message. */
+#ifdef WIN32
+_declspec(dllexport)
+#endif
+void md5_append(md5_state_t *pms, const md5_byte_t *data, int nbytes);
+
+/* Finish the message and return the digest. */
+#ifdef WIN32
+_declspec(dllexport)
+#endif
+void md5_finish(md5_state_t *pms, md5_byte_t digest[16]);
+void md5_digest(const char* inString, unsigned char md5pword[16]);
+unsigned int hash_key(const char* inString);
+#ifdef __cplusplus
+}  /* end extern "C" */
+#endif
+
+#endif /* md5_INCLUDED */
diff -ruN memcache-3.0.4/memcache.c memcache-3.0.4-ketama/memcache.c
--- memcache-3.0.4/memcache.c	2009-02-22 17:01:43.000000000 +0100
+++ memcache-3.0.4-ketama/memcache.c	2009-12-31 11:40:59.000000000 +0100
@@ -180,8 +180,11 @@
 	else if (!strcasecmp(new_value, "consistent")) {
 		MEMCACHE_G(hash_strategy) = MMC_CONSISTENT_HASH;
 	}
+	else if (!strcasecmp(new_value, "ketama")) {
+		MEMCACHE_G(hash_strategy) = MMC_KETAMA_HASH;
+	}
 	else {
-		php_error_docref(NULL TSRMLS_CC, E_WARNING, "memcache.hash_strategy must be in set {standard, consistent} ('%s' given)", new_value);
+		php_error_docref(NULL TSRMLS_CC, E_WARNING, "memcache.hash_strategy must be in set {standard, consistent, ketama} ('%s' given)", new_value);
 		return FAILURE;
 	}
 
@@ -197,8 +200,11 @@
 	else if (!strcasecmp(new_value, "fnv")) {
 		MEMCACHE_G(hash_function) = MMC_HASH_FNV1A;
 	}
+	else if (!strcasecmp(new_value, "md5")) {
+		MEMCACHE_G(hash_function) = MMC_HASH_MD5;
+	}
 	else {
-		php_error_docref(NULL TSRMLS_CC, E_WARNING, "memcache.hash_function must be in set {crc32, fnv} ('%s' given)", new_value);
+		php_error_docref(NULL TSRMLS_CC, E_WARNING, "memcache.hash_function must be in set {crc32, fnv, md5} ('%s' given)", new_value);
 		return FAILURE;
 	}
 
diff -ruN memcache-3.0.4/memcache_ketama_hash.c memcache-3.0.4-ketama/memcache_ketama_hash.c
--- memcache-3.0.4/memcache_ketama_hash.c	1970-01-01 01:00:00.000000000 +0100
+++ memcache-3.0.4-ketama/memcache_ketama_hash.c	2009-12-31 11:25:59.000000000 +0100
@@ -0,0 +1,209 @@
+/*
+  +----------------------------------------------------------------------+
+  | PHP Version 5                                                        |
+  +----------------------------------------------------------------------+
+  | Copyright (c) 1997-2007 The PHP Group                                |
+  +----------------------------------------------------------------------+
+  | This source file is subject to version 3.0 of the PHP license,       |
+  | that is bundled with this package in the file LICENSE, and is        |
+  | available through the world-wide-web at the following url:           |
+  | http://www.php.net/license/3_0.txt.                                  |
+  | If you did not receive a copy of the PHP license and are unable to   |
+  | obtain it through the world-wide-web, please send a note to          |
+  | license@php.net so we can mail you a copy immediately.               |
+  +----------------------------------------------------------------------+
+  | Author: Marlon de Boer <marlon AT hyves DOT nl>                      |
+  +----------------------------------------------------------------------+
+*/
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include <stdlib.h>
+#include <math.h>
+
+#include "php.h"
+#include "php_memcache.h"
+#include "md5.h"
+
+ZEND_EXTERN_MODULE_GLOBALS(memcache)
+
+struct mcs {
+	mmc_t					*server;
+	unsigned int			point;
+};
+
+struct mmc_servers {
+	mmc_t					*server;
+	int						weight;
+	struct mmc_servers		*next;
+};
+
+typedef struct mmc_ketama_state {
+	int						num_servers;
+	struct mmc_servers		*servers;
+	int						cont_points;
+	unsigned int			total_weight;
+	mmc_hash_function_t		*hash;
+} mmc_ketama_state_t;
+
+struct mcs *continuum = NULL;
+
+void *mmc_ketama_create_state(mmc_hash_function_t *hash) /* {{{ */
+{
+	mmc_ketama_state_t *state = emalloc(sizeof(mmc_ketama_state_t));
+	memset(state, 0, sizeof(mmc_ketama_state_t));
+	
+	state->hash = hash;
+	state->servers = NULL;
+	
+	return state;
+}
+/* }}} */
+
+void mmc_ketama_free_state(void *s) /* {{{ */
+{
+	mmc_ketama_state_t *state = s;
+	struct mmc_servers *ptr = NULL, *ptr_old = NULL;
+
+	if (state->servers != NULL) {
+		ptr = state->servers;
+		ptr_old = ptr;
+
+		while (ptr->next != NULL) {
+			ptr_old = ptr;
+			ptr = ptr->next;
+			efree(ptr_old);
+		}
+
+		efree(ptr);
+	}
+
+	if (continuum != NULL)
+		efree(continuum);
+	
+	efree(state);
+}
+/* }}} */
+static int mmc_ketama_compare(const void *a, const void *b) /* {{{ */
+{
+	return (((struct mcs *)a)->point < ((struct mcs *)b)->point) ? -1 : (((struct mcs *)a)->point > ((struct mcs *)b)->point ? 1 : 0);
+}
+/* }}} */
+
+mmc_t *mmc_ketama_find_server(void *s, const char *key, unsigned int key_len TSRMLS_DC) /* {{{ */
+{
+	mmc_ketama_state_t *state = s;
+	unsigned int midval, midval1, cont_point = hash_key(key);
+	int midp, lowp = 0, highp = state->cont_points;
+	
+	while (1) {
+		midp = (int)( ( lowp+highp ) / 2 );
+		if (midp == state->cont_points) {
+			return continuum[0].server;
+		}
+		midval = continuum[midp].point;
+		midval1 = midp == 0 ? 0 : continuum[midp-1].point;
+
+		if (cont_point < midval && cont_point > midval1)
+			return continuum[midp].server;
+
+		if (midval < cont_point) {
+			lowp = midp + 1;
+		} else {
+			highp = midp - 1;
+		}
+
+		if (lowp > highp)
+			return continuum[0].server;
+	}
+}
+/* }}} */
+
+void mmc_ketama_add_server(void *s, mmc_t *mmc, unsigned int weight) /* {{{ */
+{
+	mmc_ketama_state_t *state = s;
+	unsigned int hash;
+
+	int k, cont_points = 0;
+	struct mmc_servers *ptr = NULL;
+
+	if (state->servers == NULL) {
+		state->servers = emalloc(sizeof(struct mmc_servers));
+		state->servers->server = mmc;
+		state->servers->weight = weight;
+		state->servers->next = NULL;
+	}
+	else {
+		ptr = state->servers;
+
+		while (ptr->next != NULL) {
+			ptr = ptr->next;
+		}
+	
+		ptr->next = emalloc(sizeof(struct mmc_servers));
+		ptr->next->server = mmc;
+		ptr->next->weight = weight;
+		ptr->next->next = NULL;
+	}
+
+	ptr = state->servers;
+	state->cont_points = cont_points;
+	state->total_weight += weight;
+	state->num_servers++;
+
+	continuum = erealloc(continuum, state->num_servers  * 160 * sizeof(struct mcs));
+	unsigned char digest[16];
+	int t;
+
+	for (t = 1; t <= state->num_servers; t++) {
+		float pct = (float) ptr->weight / (float) state->total_weight;
+		unsigned int ks = (unsigned int)floorf(pct * 40.0 * (float) state->num_servers);
+		unsigned char digest[16];
+		int ss_length, h;
+		
+		for (k = 0; k < ks; k++)
+		{
+			/* max length string: hostname + port + ks + \0 */
+			ss_length = strlen(ptr->server->host) + sizeof(int) * 8 + 2 + 1;
+			char *ss = emalloc(ss_length);
+			sprintf(ss, "%s:%d-%d", ptr->server->host, ptr->server->tcp.port, k);
+			ss[ss_length-1] = '\0';
+			md5_digest(ss, digest);
+			
+			for (h = 0; h < 4; h++) {
+				continuum[cont_points].point = (unsigned int)( digest[3+h*4] << 24 )
+										| ( digest[2+h*4] << 16 )
+										| ( digest[1+h*4] <<  8 )
+										| digest[h*4];
+				continuum[cont_points].server = ptr->server;
+				cont_points++;
+			}
+			efree(ss);
+		}
+
+		ptr = ptr->next;
+	}
+
+	state->cont_points = cont_points;
+	qsort((void*) continuum, cont_points, sizeof(struct mcs), mmc_ketama_compare);
+}
+
+/* }}} */
+
+mmc_hash_strategy_t mmc_ketama_hash = {
+	mmc_ketama_create_state,
+	mmc_ketama_free_state,
+	mmc_ketama_find_server,
+	mmc_ketama_add_server
+};
+
+/*
+ * Local variables:
+ * tab-width: 4
+ * c-basic-offset: 4
+ * End:
+ * vim600: noet sw=4 ts=4 fdm=marker
+ * vim<600: noet sw=4 ts=4
+ */
diff -ruN memcache-3.0.4/memcache_pool.c memcache-3.0.4-ketama/memcache_pool.c
--- memcache-3.0.4/memcache_pool.c	2009-12-31 11:44:11.000000000 +0100
+++ memcache-3.0.4-ketama/memcache_pool.c	2009-12-31 11:33:15.000000000 +0100
@@ -33,6 +33,7 @@
 #include "ext/standard/php_string.h"
 #include "ext/standard/php_smart_str.h"
 #include "memcache_pool.h"
+#include "md5.h"
 
 ZEND_DECLARE_MODULE_GLOBALS(memcache)
 
@@ -53,6 +54,22 @@
 }
 /* }}} */
 
+static unsigned int mmc_hash_md5_init()						{ return 0; }
+static unsigned int mmc_hash_md5_finish(unsigned int seed)	{ return seed; }
+
+static unsigned int mmc_hash_md5_combine(unsigned int seed, const void *key, unsigned int key_len) /*
+	MD5 hash {{{ */
+{
+	return hash_key((const char *)key);
+}
+/* }}} */
+
+mmc_hash_function_t mmc_hash_md5 = {
+	mmc_hash_md5_init,
+	mmc_hash_md5_combine,
+	mmc_hash_md5_finish
+};
+
 static unsigned int mmc_hash_crc32_init()						{ return ~0; }
 static unsigned int mmc_hash_crc32_finish(unsigned int seed)	{ return ~seed; }
 
@@ -844,6 +861,9 @@
 		case MMC_CONSISTENT_HASH:
 			pool->hash = &mmc_consistent_hash;
 			break;
+		case MMC_KETAMA_HASH:
+			pool->hash = &mmc_ketama_hash;
+			break;
 		default:
 			pool->hash = &mmc_standard_hash;
 	}
@@ -852,6 +872,9 @@
 		case MMC_HASH_FNV1A:
 			hash = &mmc_hash_fnv1a;
 			break;
+		case MMC_HASH_MD5:
+			hash = &mmc_hash_md5;
+			break;
 		default:
 			hash = &mmc_hash_crc32;
 	}
diff -ruN memcache-3.0.4/memcache_pool.h memcache-3.0.4-ketama/memcache_pool.h
--- memcache-3.0.4/memcache_pool.h	2009-12-31 11:44:12.000000000 +0100
+++ memcache-3.0.4-ketama/memcache_pool.h	2009-12-31 11:29:40.000000000 +0100
@@ -92,8 +92,10 @@
 
 #define MMC_STANDARD_HASH 		1
 #define MMC_CONSISTENT_HASH 	2
+#define MMC_KETAMA_HASH			3
 #define MMC_HASH_CRC32 			1			/* CRC32 hash function */
 #define MMC_HASH_FNV1A 			2			/* FNV-1a hash function */
+#define MMC_HASH_MD5			3			/* MD5 hash function */
 
 #define MMC_CONSISTENT_POINTS	160			/* points per server */
 #define MMC_CONSISTENT_BUCKETS	1024		/* number of precomputed buckets, should be power of 2 */
@@ -272,6 +274,7 @@
 
 extern mmc_hash_function_t mmc_hash_crc32;
 extern mmc_hash_function_t mmc_hash_fnv1a;
+extern mmc_hash_function_t mmc_hash_md5;
 
 #define mmc_hash(hash, key, key_len) ((hash)->finish((hash)->combine((hash)->init(), (key), (key_len))))
 
@@ -289,6 +292,7 @@
 
 extern mmc_hash_strategy_t mmc_standard_hash;
 extern mmc_hash_strategy_t mmc_consistent_hash;
+extern mmc_hash_strategy_t mmc_ketama_hash;
 
 /* 32 bit magic FNV-1a prime and init */
 #define FNV_32_PRIME	0x01000193


Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2009-12-31 07:53 UTC] tony at daylessday dot org
What's the point in bundling our own MD5 implementation if we already have one in PHP? (we can use it in the extension, of course)
 [2010-03-25 04:54 UTC] pierre dot php at gmail dot com
Agreed, can you update your patch to use the PHP MD5 implementation please?
 [2011-09-27 23:02 UTC] nadine at nadineshenton dot com
Holy Toledo, so glad I clicekd on this site first!
 [2011-10-02 10:54 UTC] dlqpzp at oiobck dot com
aINept  <a href="http://yljehrqtjyde.com/">yljehrqtjyde</a>
 [2013-02-18 00:35 UTC] pecl-dev at lists dot php dot net
No feedback was provided. The bug is being suspended because
we assume that you are no longer experiencing the problem.
If this is not the case and you are able to provide the
information that was requested earlier, please do so and
change the status of the bug back to "Open". Thank you.
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Thu Mar 28 17:01:29 2024 UTC