|  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #47526 PCRE fails on Unicode surrogates
Submitted: 2009-02-28 08:51 UTC Modified: 2009-04-10 16:16 UTC
From: phpwnd at gmail dot com Assigned:
Status: Not a bug Package: PCRE related
PHP Version: 5.3CVS-2009-02-28 (CVS) OS: *
Private report: No CVE-ID: None
View Add Comment Developer Edit
Anyone can comment on a bug. Have a simpler test case? Does it work for you on a different platform? Let us know!
Just going to say 'Me too!'? Don't clutter the database with that please !
Your email address:
Solve the problem:
49 - 32 = ?
Subscribe to this entry?

 [2009-02-28 08:51 UTC] phpwnd at gmail dot com
According to PCRE functions should be able to match surrogates in Unicode mode. However, it is my understanding that surrogates are not allowed in UTF-8, which is the encoding used by the Unicode mode. That would explain why preg_match() and preg_replace() fail when operating on UTF-8-encoded surrogates.

Note that both functions fail in a different way. preg_match() returns 0 whereas preg_replace() returns NULL.

I'm not sure what the fix should be. Being able to match surrogates would make my life easier, but if it's not valid UTF-8 then it might be more consistent (albeit in a twisted way) to return NULL, as that's what PCRE functions do on invalid UTF-8.

Reproduce code:
// \xED\xA0\x80 is character 0xD800 in UTF-8
var_dump(preg_match('#.#u', ".\xED\xA0\x80"));
var_dump(preg_replace('#\p{Cs}#u', '', ".\xED\xA0\x80"));

Expected result:
string(1) "."

Actual result:


Add a Patch

Pull Requests

Add a Pull Request


AllCommentsChangesGit/SVN commitsRelated reports
 [2009-04-10 15:58 UTC]
As far as I understand that codepoint is invalid in UTF-8.
If you call preg_last_error() after preg_match() it will return PREG_BAD_UTF8_ERROR, confirming my hipothesis.
So no bug here.
 [2009-04-10 16:16 UTC] phpwnd at gmail dot com
My point exactly. Why do we have an escape sequence for surrogates when they are invalid and it doesn't work anyway? \p{Cs} appears in the manual ( under "Supported property codes"

Also, why do preg_match() and preg_replace() fail differently? preg_match returns 0, which lets the user believe the input was valid but didn't match, whereas preg_replace() returns NULL, which indicates the input was invalid. I cannot verify what preg_last_error() says right now as I'm having trouble with latest CVS.
PHP Copyright © 2001-2022 The PHP Group
All rights reserved.
Last updated: Tue May 24 09:05:44 2022 UTC