PHP :: Bug #74878 :: Data race in ZTS builds

Bug #74878

Data race in ZTS builds

Submitted:

2017-07-07 20:11 UTC

Modified:

2017-09-17 07:28 UTC

Votes:	2
Avg. Score:	5.0 ± 0.0
Reproduced:	1 of 1 (100.0%)
Same Version:	1 (100.0%)
Same OS:	1 (100.0%)

From:

maroszek at gmx dot net

Assigned:

dmitry (profile)

Status:

Closed

Package:

*General Issues

PHP Version:

7.2.0alpha3

OS:

Linux/MacOS

Private report:

CVE-ID:

None

View Developer Edit

[2017-07-07 20:11 UTC] maroszek at gmx dot net

Description:
------------
Build PHP with thread sanitizer:
CFLAGS="-O1 -g -fsanitize=thread" ./configure --disable-fpm --disable-cgi --disable-cli --disable-phpdbg --enable-embed=static --enable-maintainer-zts --without-iconv --enable-json --enable-mbstring

Use my ZTS Embed example which produces a lot of parallel executed scripts.
https://gist.github.com/paresy/3cbd4c6a469511ac7479aa0e7c42fea7

Build it:
clang++ crash.cpp -Imain -ITSRM -IZend -I. --std=c++11 -Llibs -lphp7 -lxml2 -lresolv -ldl -pthread -fsanitize=thread -g -O1

Run it:
./a.out

Background:
I am trying to locate random segfaults in PHP 7.2 since two weeks... and this is the best i could find yet... (It is a lot better since 7.2! Thanks a lot!) Valgrind shows nothing... And this only shows data races... but i cannot reproduce a crash :( Anyway... It should be fixed nevertheless. Hopefully the data is useful :)

Hint:
Sometimes the data race happens in zend_destroy_hash. See second log from the output.

Test script:
---------------
<?

	class JSONRPCException extends Exception {
		//
	}

	class JSONRPC {

		private $url;
		private $username;
		private $password;

		function __construct($url) {
			$data = @parse_url($url);
			
			if(($data === false) || !isset($data['scheme']) || !isset($data['host']))
				throw new Exception("Invalid URL");
				
			$this->url = $data['scheme']."://".$data['host'];
			if(isset($data['port']))
				$this->url .= ":".$data['port'];
			if(isset($data['path']))
				$this->url .= $data['path'];
			
			if(isset($data['user']))
				$this->username = $data['user'];
			else
				$this->username = "";
				
			if(isset($data['pass']))
				$this->password = $data['pass'];
			else
				$this->password = "";
		}

		public function __call($method, $arguments) {
			return self::makeRequest($this->url, $this->username, $this->password, $method, $arguments, false);
		}

		static public function __callStatic($method, $params) {
			return self::makeRequest("http://127.0.0.1:3777/api/", "", "", $method, $params, true);
		}

		static private function makeRequest($url, $username, $password, $method, $params, $customErrorHandler) {
				
			try {
			
				if (!is_scalar($method)) {
					throw new Exception('Method name has no scalar value');
				}
				
				if (!is_array($params)) {
					throw new Exception('Params must be given as array');
				}

				$id = round(fmod(microtime(true)*1000, 10000));
				$params = array_values($params);
				$strencode = function(&$item, $key) {
					if ( is_string($item) )
						$item = utf8_encode($item);
					else if ( is_array($item) )
						array_walk_recursive($item, $strencode);
				};
				array_walk_recursive($params, $strencode);
				
				$request = Array(
									"jsonrpc" => "2.0",
									"method" => $method,
									"params" => $params,
									"id" => $id
								);
								
				$request = json_encode($request);
				
				$header = "Content-type: application/json"."\r\n";
				if(($username != "") || ($password != "")) {
					$header .= "Authorization: Basic ".base64_encode($username.":".$password)."\r\n";
				}
			
				$options = Array(
									"http" => array (
														"method"  => 'POST',
														"header"  => $header,
														"content" => $request
													)
								);
				
				$context  = stream_context_create($options);		
				$response = file_get_contents($url, false, $context);

				if($response === false)
				{
					throw new Exception('Unable to connect');
				}

				if((strpos($http_response_header[0], "200") === false)) {
					throw new Exception('Server did not respond successfully');
				}
					
				$response = json_decode($response, true);
							
				if (is_null($response)) {
					throw new Exception('Request error: No response');
				}
				
				$strdecode = function(&$item, $key) {
					if ( is_string($item) )
						$item = utf8_decode($item);
					else if ( is_array($item) )
						array_walk_recursive($item, $strdecode);
				};
				array_walk_recursive($response, $strdecode);
				
				if (isset($response['error'])) {
					throw new JSONRPCException($response['error']['message']);
				}

				if (!isset($response['id'])) {
					throw new Exception('No response id');
				} elseif ($response['id'] != $id) {
						throw new Exception('Incorrect response id (request id: ' . $id . ', response id: ' . $response['id'] . ')');
				}		
					
				return $response['result'];
				
			} catch (Exception $e) {
				
				if($customErrorHandler) {
					set_error_handler(array('JSONRPC', '__errorHandler'));
					$trace = $e->getTrace();
					trigger_error($e->getMessage().' in '.$trace[3]['file'].' on line '.$trace[3]['line'], E_USER_WARNING);
					restore_error_handler();
					return false;
				} else {
					throw $e;
				}
				
			}			
				
		}		
		
		static private function __errorHandler($errno, $errstr, $errfile, $errline)
		{
			if (!(error_reporting() & $errno)) {
				return;
			}
			echo $errstr;
			return true;
		}		
		
	}

?>

Expected result:
----------------
No output. 

Actual result:
--------------
==================
WARNING: ThreadSanitizer: data race (pid=25837)
  Write of size 4 at 0x7d0800009560 by thread T2:
    #0 do_inherit_property zend_inheritance.c:709 (a.out+0x000100556bb4)
    #1 zend_do_inheritance zend_inheritance.c:925 (a.out+0x0001005565b1)
    #2 do_bind_inherited_class zend_compile.c:1183 (a.out+0x0001004a7bcd)
    #3 zend_do_early_binding zend_compile.c:1240 (a.out+0x0001004a8279)
    #4 zend_compile_top_stmt zend_compile.c:8080 (a.out+0x0001004bf1d1)
    #5 zend_compile_top_stmt zend_compile.c:8068 (a.out+0x0001004bf116)
    #6 zend_compile zend_language_scanner.l:601 (a.out+0x000100468c77)
    #7 compile_file zend_language_scanner.l:635 (a.out+0x0001004688bf)
    #8 phar_compile_file phar.c:3320 (a.out+0x0001002d76d5)
    #9 zend_execute_scripts zend.c:1484 (a.out+0x0001004eb4c9)
    #10 php_execute_script main.c:2550 (a.out+0x00010042c815)
    #11 main::$_0::operator()() const crash.cpp:35 (a.out+0x000100001ae3)
    #12 void* std::__1::__thread_proxy<std::__1::tuple<main::$_0> >(void*) thread:357 (a.out+0x0001000019c2)

  Previous write of size 4 at 0x7d0800009560 by thread T1:
    #0 do_inherit_property zend_inheritance.c:709 (a.out+0x000100556bb4)
    #1 zend_do_inheritance zend_inheritance.c:925 (a.out+0x0001005565b1)
    #2 do_bind_inherited_class zend_compile.c:1183 (a.out+0x0001004a7bcd)
    #3 zend_do_early_binding zend_compile.c:1240 (a.out+0x0001004a8279)
    #4 zend_compile_top_stmt zend_compile.c:8080 (a.out+0x0001004bf1d1)
    #5 zend_compile_top_stmt zend_compile.c:8068 (a.out+0x0001004bf116)
    #6 zend_compile zend_language_scanner.l:601 (a.out+0x000100468c77)
    #7 compile_file zend_language_scanner.l:635 (a.out+0x0001004688bf)
    #8 phar_compile_file phar.c:3320 (a.out+0x0001002d76d5)
    #9 zend_execute_scripts zend.c:1484 (a.out+0x0001004eb4c9)
    #10 php_execute_script main.c:2550 (a.out+0x00010042c815)
    #11 main::$_0::operator()() const crash.cpp:35 (a.out+0x000100001ae3)
    #12 void* std::__1::__thread_proxy<std::__1::tuple<main::$_0> >(void*) thread:357 (a.out+0x0001000019c2)

  Location is heap block of size 32 at 0x7d0800009560 allocated by main thread:
    #0 malloc <null>:144 (libclang_rt.tsan_osx_dynamic.dylib+0x00000004188a)
    #1 __zend_malloc zend_alloc.c:2811 (a.out+0x0001004a213a)
    #2 zend_declare_property zend_API.c:3766 (a.out+0x0001004fa0fe)
    #3 zend_declare_property_string zend_API.c:3814 (a.out+0x0001004fa47a)
    #4 zend_register_default_exception zend_exceptions.c:820 (a.out+0x00010052866c)
    #5 zend_register_default_classes zend_default_classes.c:34 (a.out+0x0001005558f6)
    #6 zm_startup_core zend_builtin_functions.c:302 (a.out+0x000100513bf1)
    #7 zend_startup_module_ex zend_API.c:1873 (a.out+0x0001004f39e6)
    #8 zend_startup_module_zval zend_API.c:1888 (a.out+0x0001004f41e1)
    #9 zend_hash_apply zend_hash.c:1506 (a.out+0x00010050dda6)
    #10 zend_startup_modules zend_API.c:1999 (a.out+0x0001004f4007)
    #11 php_module_startup main.c:2269 (a.out+0x000100429f94)
    #12 php_embed_startup php_embed.c:109 (a.out+0x0001006385f1)
    #13 php_embed_init php_embed.c:200 (a.out+0x0001006388d2)
    #14 main crash.cpp:10 (a.out+0x000100001209)

  Thread T2 (tid=2628148, running) created by main thread at:
    #0 pthread_create <null>:144 (libclang_rt.tsan_osx_dynamic.dylib+0x000000024410)
    #1 std::__1::thread::thread<main::$_0, void>(main::$_0&&) thread:369 (a.out+0x000100001936)
    #2 std::__1::thread::thread<main::$_0, void>(main::$_0&&) thread:365 (a.out+0x0001000018f0)
    #3 std::__1::shared_ptr<std::__1::thread> std::__1::shared_ptr<std::__1::thread>::make_shared<main::$_0>(main::$_0&&) memory:3800 (a.out+0x0001000017b1)
    #4 main crash.cpp:14 (a.out+0x00010000137a)

  Thread T1 (tid=2628147, running) created by main thread at:
    #0 pthread_create <null>:144 (libclang_rt.tsan_osx_dynamic.dylib+0x000000024410)
    #1 std::__1::thread::thread<main::$_0, void>(main::$_0&&) thread:369 (a.out+0x000100001936)
    #2 std::__1::thread::thread<main::$_0, void>(main::$_0&&) thread:365 (a.out+0x0001000018f0)
    #3 std::__1::shared_ptr<std::__1::thread> std::__1::shared_ptr<std::__1::thread>::make_shared<main::$_0>(main::$_0&&) memory:3800 (a.out+0x0001000017b1)
    #4 main crash.cpp:14 (a.out+0x00010000137a)

SUMMARY: ThreadSanitizer: data race zend_inheritance.c:709 in do_inherit_property

WARNING: ThreadSanitizer: data race (pid=50889)
  Write of size 4 at 0x7d0800009560 by thread T1:
    #0 zend_hash_destroy zend_hash.c:1268 (a.out+0x00010050cbcc)
    #1 destroy_zend_class zend_opcode.c:309 (a.out+0x0001004d40a6)
    #2 zend_hash_reverse_apply zend_hash.c:997 (a.out+0x00010050f926)
    #3 shutdown_executor zend_execute_API.c:375 (a.out+0x0001004ca8dc)
    #4 zend_deactivate zend.c:1060 (a.out+0x0001004e9ac6)
    #5 php_request_shutdown main.c:1879 (a.out+0x000100428dc1)
    #6 main::$_0::operator()() const crash.cpp:31 (a.out+0x0001000016ef)
    #7 void* std::__1::__thread_proxy<std::__1::tuple<main::$_0> >(void*) thread:357 (a.out+0x0001000015c2)

  Previous write of size 4 at 0x7d0800009560 by thread T3:
    #0 do_inherit_property zend_inheritance.c:709 (a.out+0x0001005575d4)
    #1 zend_do_inheritance zend_inheritance.c:925 (a.out+0x000100556fd1)
    #2 do_bind_inherited_class zend_compile.c:1181 (a.out+0x0001004a79fd)
    #3 zend_do_early_binding zend_compile.c:1238 (a.out+0x0001004a80a9)
    #4 zend_compile_top_stmt zend_compile.c:8073 (a.out+0x0001004befa1)
    #5 zend_compile_top_stmt zend_compile.c:8061 (a.out+0x0001004beee6)
    #6 zend_compile zend_language_scanner.l:601 (a.out+0x000100468aa7)
    #7 compile_file zend_language_scanner.l:635 (a.out+0x0001004686ef)
    #8 phar_compile_file phar.c:3320 (a.out+0x0001002d7715)
    #9 zend_execute_scripts zend.c:1531 (a.out+0x0001004eba39)
    #10 php_execute_script main.c:2548 (a.out+0x00010042c715)
    #11 main::$_0::operator()() const crash.cpp:35 (a.out+0x0001000016e3)
    #12 void* std::__1::__thread_proxy<std::__1::tuple<main::$_0> >(void*) thread:357 (a.out+0x0001000015c2)

  Location is heap block of size 32 at 0x7d0800009560 allocated by main thread:
    #0 malloc <null>:144 (libclang_rt.tsan_osx_dynamic.dylib+0x00000004188a)
    #1 __zend_malloc zend_alloc.c:2811 (a.out+0x0001004a1f6a)
    #2 zend_declare_property zend_API.c:3746 (a.out+0x0001004fc30e)
    #3 zend_declare_property_string zend_API.c:3794 (a.out+0x0001004fc68a)
    #4 zend_register_default_exception zend_exceptions.c:820 (a.out+0x000100528d1c)
    #5 zend_register_default_classes zend_default_classes.c:34 (a.out+0x000100556316)
    #6 zm_startup_core zend_builtin_functions.c:302 (a.out+0x000100514301)
    #7 zend_startup_module_ex zend_API.c:1873 (a.out+0x0001004f3f56)
    #8 zend_startup_module_zval zend_API.c:1888 (a.out+0x0001004f4751)
    #9 zend_hash_apply zend_hash.c:1507 (a.out+0x00010050e4b6)
    #10 zend_startup_modules zend_API.c:1999 (a.out+0x0001004f4577)
    #11 php_module_startup main.c:2269 (a.out+0x000100429e94)
    #12 php_embed_startup php_embed.c:109 (a.out+0x0001006396b1)
    #13 php_embed_init php_embed.c:200 (a.out+0x000100639992)
    #14 main crash.cpp:10 (a.out+0x000100000e09)

  Thread T1 (tid=2489677, running) created by main thread at:
    #0 pthread_create <null>:144 (libclang_rt.tsan_osx_dynamic.dylib+0x000000024410)
    #1 std::__1::thread::thread<main::$_0, void>(main::$_0&&) thread:369 (a.out+0x000100001536)
    #2 std::__1::thread::thread<main::$_0, void>(main::$_0&&) thread:365 (a.out+0x0001000014f0)
    #3 std::__1::shared_ptr<std::__1::thread> std::__1::shared_ptr<std::__1::thread>::make_shared<main::$_0>(main::$_0&&) memory:3800 (a.out+0x0001000013b1)
    #4 main crash.cpp:14 (a.out+0x000100000f7a)

  Thread T3 (tid=2489679, running) created by main thread at:
    #0 pthread_create <null>:144 (libclang_rt.tsan_osx_dynamic.dylib+0x000000024410)
    #1 std::__1::thread::thread<main::$_0, void>(main::$_0&&) thread:369 (a.out+0x000100001536)
    #2 std::__1::thread::thread<main::$_0, void>(main::$_0&&) thread:365 (a.out+0x0001000014f0)
    #3 std::__1::shared_ptr<std::__1::thread> std::__1::shared_ptr<std::__1::thread>::make_shared<main::$_0>(main::$_0&&) memory:3800 (a.out+0x0001000013b1)
    #4 main crash.cpp:14 (a.out+0x000100000f7a)

SUMMARY: ThreadSanitizer: data race zend_hash.c:1268 in zend_hash_destroy

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports

[2017-07-07 20:24 UTC] nikic@php.net

-Status: Open +Status: Analyzed

[2017-07-07 20:24 UTC] nikic@php.net

The problem is that https://github.com/php/php-src/blob/master/Zend/zend_API.c#L3758 uses the original `name` instead of the interned `property_info->name`.

[2017-07-07 20:32 UTC] nikic@php.net

Actually it can't use property_info->name as the key is supposed to be the non-mangled name. So it needs to be interned separately.

[2017-07-07 20:42 UTC] nikic@php.net

Automatic comment on behalf of nikita.ppv@gmail.com
Revision: http://git.php.net/?p=php-src.git;a=commit;h=9069734b57a556a23f977915ee2819477c6d6720
Log: Fixed bug #74878

[2017-07-07 20:42 UTC] nikic@php.net

-Status: Analyzed +Status: Closed

[2017-07-10 09:22 UTC] dmitry@php.net

Automatic comment on behalf of dmitry@zend.com
Revision: http://git.php.net/?p=php-src.git;a=commit;h=161c378cc861fe01ea0034f5a6e8d32ad14f5ccb
Log: Revert &quot;Fixed bug #74878&quot;

[2017-07-10 14:19 UTC] maroszek at gmx dot net

Hi,

i have seen the commit got reverted. Any hints why? Can we reopen the issue then?

The previous commit successfully fixed the crashing issues i was experiencing.

Thanks!

[2017-07-10 14:22 UTC] spam2 at rhsoft dot net

because it created more problems than it solves

-------- Weitergeleitete Nachricht --------
Betreff: Re: [PHP-CVS] com php-src: Fixed bug #74878: NEWS Zend/zend_API.c
Datum: Mon, 10 Jul 2017 08:08:22 +0000
Von: Dmitry Stogov <dmitry@zend.com>
An: Nikita Popov <nikic@php.net>, php-cvs@lists.php.net <php-cvs@lists.php.net>
Kopie (CC): Sara Golemon <pollita@php.net>

This commit introduced a lot of memory leaks.

Probably, you meant string interning only for internal classes.

Even for internal classes only, this may require additional changes in opcache, that copies interned strings into shared memory.


It's better to revert this, before complete fix

[2017-07-10 14:22 UTC] spam2 at rhsoft dot net

Related To: Bug #74878

[2017-07-10 14:50 UTC] maroszek at gmx dot net

Thanks! Didn't knew where to look :)

If i can somehow assist in fully solving the problem, i'll gladly do that!

[2017-07-18 09:29 UTC] maroszek at gmx dot net

Hi! 

I just wanted to kindly ask if this is still on your radar as the issue is marked as closed?

Thanks!

[2017-07-18 09:56 UTC] nikic@php.net

-Status: Closed +Status: Re-Opened

[2017-08-07 13:51 UTC] maroszek at gmx dot net

Hi,

i'd love to bump this one. I think this is the only remaining blocker issue for ZTS usage on PHP 7.2. Using the incomplete patch from nikic fixes all crash related issues for my use case. (I know the memory leak problems) It would be awesome if we could get this properly fixed and tested until the 7.2 release. I would love to help - but i have no idea where to start.

Would it help if i could reproduce the issue with the pthreads extension? (https://github.com/krakjoe/pthreads)? It should be affected aswell.

Thanks a lot in advance!

[2017-08-19 00:36 UTC] ab@php.net

Related To: Bug #75048

[2017-09-17 05:31 UTC] maroszek at gmx dot net

Hi,

i'd like to make a quick followup... did the problematic fix from nikic (http://git.php.net/?p=php-src.git;a=blobdiff;f=Zend/zend_API.c;h=609e8b4a515a3ac71c6ea1d720a9941229d8f9da;hp=ca8b4e2f9974f43ce4e3105f374410d8bcc4fc34;hb=9069734b57a556a23f977915ee2819477c6d6720;hpb=134047dfde1feaf664b54bb2f84324a38eced091) only leak with opcache extension enabled? Or was it a general issue?

I am still using the "broken" patch (no opcache extension) and the crashes are gone.

Thanks,
Michael

[2017-09-17 07:28 UTC] nikic@php.net

-Assigned To: +Assigned To: dmitry

[2017-09-17 07:28 UTC] nikic@php.net

I haven't seen the leaks myself, so assigning this to @dmitry, who raised the issue.

[2017-09-18 08:28 UTC] dmitry@php.net

Automatic comment on behalf of dmitry@zend.com
Revision: http://git.php.net/?p=php-src.git;a=commit;h=41e5f916bffaf59b3a2374511e45cb7a22bb2665
Log: Fixed bug #74878 (Data race in ZTS builds)

[2017-09-18 08:28 UTC] dmitry@php.net

-Status: Re-Opened +Status: Closed

[2017-10-16 15:53 UTC] ab@php.net

Related To: Bug #75048

	php.net \| support \| documentation \| report a bug \| advanced search \| search howto \| statistics \| random bug \| login
go to bug id or search bugs for


Copyright © 2001-2024 The PHP Group All rights reserved.	Last updated: Thu Nov 21 11:01:29 2024 UTC