php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Request #45650 MD5 should use utf8 to be comparable with non-php hashes
Submitted: 2008-07-29 02:39 UTC Modified: 2008-07-29 12:33 UTC
From: tom at tdwright dot co dot uk Assigned:
Status: Not a bug Package: Feature/Change Request
PHP Version: 5.2.6 OS: Linux + Windows
Private report: No CVE-ID: None
View Developer Edit
 [2008-07-29 02:39 UTC] tom at tdwright dot co dot uk 
Description:
------------
It seems that the MD5 function in php uses the UTF7 encoding of a string for the algorithm. Every other implementation seems to use UTF8.
Finding out about this discrepancy was not easy as
a) The programming community at large presumably takes it for granted that MD5 uses a UTF8 encoded string
b) PHP programmers don't often need to compare their PHP generated hashes with those generated outside of PHP.
It's a really annoying quirk and I'd love to see a change (even if it's an option).

Reproduce code:
---------------
<?php
$hash_from_another_lang = $_POST['hash1'];
$php_hash = md5("hashtext");
return ($hash_from_another_lang == $php_hash);
?>

Expected result:
----------------
true

MD5 hashes should match wherever they are generated.

Actual result:
--------------
false

The PHP implementation of the MD5 algorithm produces hashes which are incongruent with the results of any other (AFAIK) MD5 implementation.

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2008-07-29 04:57 UTC] moriyoshi@php.net 
Thank you for taking the time to write to us, but this is not
a bug. Please double-check the documentation available at
http://www.php.net/manual/ and the instructions on how to report
a bug at http://bugs.php.net/how-to-report.php

md5 always takes the argument as a bit vector rather than a string of 
letters, i.e. no encoding matters. If your script is written in ISO-
8559-15 and you passed an embedded string literal to md5(), the result 
is the hash of a ISO-8859-15 string, even though the script accepts HTTP 
requests that are supposed to be in UTF-8.
 [2008-07-29 11:34 UTC] tom at tdwright dot co dot uk 
OK, so if the encoding doesn't matter, something else is wrong.
Take a look at this script:
http://tdwright.co.uk/phpplayground/scribesense/pollscript.php?hash=ecb38fcfc2a18b712ed3dea22a3a65e7
It takes a hashed key/lock pair generated by a remote non-php client and compares it to a hash of the same string produced locally by php.
For debugging purposes I've told it to output the string to be hashed, the local hash and the supplied hash.
Note that the hash generated by two different online md5 generators (http://www.miraclesalad.com/webtools/md5.php and http://md5.br-design.co.uk/) both agree with the value produced by the non-php MD5.
You've marked this as bogus because I attributed the fault incorrectly, but that doesn't mean MD5 works properly and I maintain that this is a bug.
Your quick response however, was certainly appreciated as I've been pulling my hair out! =)
 [2008-07-29 11:48 UTC] tom at tdwright dot co dot uk 
</stupidity>
Scratch all of the above.
And please accept my most sincere apologies for wasting your generously donated time.
 [2008-07-29 11:55 UTC] derick@php.net 
Maybe you want to add what the problem was, so that others can find that possibly here and don't waste time in the future :)
 [2008-07-29 12:33 UTC] tom at tdwright dot co dot uk 
Naturally...
OK, so part of the lock/key string I was hashing was from a static file read with fopen. Unbeknown to me, the string that was read contained trailing whitespace which affected the hash.
Bit of a d'oh moment when I reversed the lock+key to key+lock and saw a space in the middle.
Easily rectified by changing my source to:
$mangle = str_replace(array("\n", "\r", "\t", " ", "\o", "\xOB"), '', $key . $lock);
$hash1 = md5($mangle);
That array of whitespace is probably a bit OTT, but I wasn't taking chances and that was a snippet I had laying around.
Anyway, my MD5 hashes now match in .Net, Flash and PHP - w00t!
 
PHP Copyright © 2001-2025 The PHP Group
All rights reserved. 		Last updated: Sun Jul 06 13:01:35 2025 UTC