PHP :: Bug #69305 :: memory leak in json

Bug #69305

memory leak in json_encode()

Submitted:

2015-03-26 11:16 UTC

Modified:

2018-11-26 21:08 UTC

Votes:	18
Avg. Score:	4.6 ± 0.7
Reproduced:	18 of 18 (100.0%)
Same Version:	3 (16.7%)
Same OS:	6 (33.3%)

From:

zoopi01 at gmail dot com

Assigned:

bukka (profile)

Status:

No Feedback

Package:

JSON related

PHP Version:

5.4.39

OS:

CentOS 5.7

Private report:

CVE-ID:

None

View Developer Edit

Welcome! If you don't have a Git account, you can't do anything here.
If you reported this bug, you can edit this bug over here.

php.net Username: php.net Password:

Quick Fix:	(description)
	Block user comment
Status:		Assign to:
Package:
Bug Type:
Summary:
From:	zoopi01 at gmail dot com
New email:
PHP Version:		OS:

New/Additional Comment:

[2015-03-26 11:16 UTC] zoopi01 at gmail dot com

Description:
------------
A few days ago, my server ran out of memory and hung up.
The server is on CentOS 5.7, Apache httpd(prefork) 2.2.22 and PHP 5.4.39.

Please see below test code for reproducing the similar situation.

After converted the json object into a json string for returning a result,
and memory usage of a httpd process was increased over than 160m and remained.
(The value of memory_limit in php.ini is 32m.)

For investigating this problem, I checked pmap and smaps output.

---------------------------------------------
# pmap -x <pid>
...
00002b6c4d374000    9364    8496    8496 rw---    [ anon ]
00002b6c4de76000  160236  154692  154692 rw---    [ anon ]
00007fff5f4b4000      84      44      44 rw---    [ stack ]
...

# cat /proc/<pid>/smaps
...
2b6c4de76000-2b6c57af1000 rw-p 2b6c4de76000 00:00 0
Size:            160236 kB
Rss:             154692 kB
Shared_Clean:         0 kB
Shared_Dirty:         0 kB
Private_Clean:        0 kB
Private_Dirty:   154692 kB
Swap:                 0 kB
Pss:             154692 kB
...
---------------------------------------------

And I traced system calls through strace. Many anonymous pages were allocated by mmap() syscall,
and munmap() syscall corresponding mmap() weren't called.
(I reproduced the situation for debugging, and some addresses are different as before.)

---------------------------------------------
# strace -p <pid> | egrep "mmap|mremap|munmap"
...
mmap(NULL, 11276288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1307754000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13087f7000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13088f7000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13089f7000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1308af7000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1308bf7000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1308cf7000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1308df7000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1308ef7000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1308ff7000
munmap(0x2b1307754000, 11276288)        = 0
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1307754000
mmap(NULL, 1183744, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1307854000
mmap(NULL, 1445888, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1307975000
mmap(NULL, 1708032, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1307ad6000
mmap(NULL, 1970176, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1307c77000
mmap(NULL, 2232320, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1307e58000
mmap(NULL, 2494464, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13090f7000
mmap(NULL, 2756608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1309358000
mmap(NULL, 3018752, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13095f9000
mmap(NULL, 3280896, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b13098da000
mmap(NULL, 3543040, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1309bfb000
mmap(NULL, 3805184, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b1309f5c000
mmap(NULL, 4067328, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b130a2fd000
mmap(NULL, 4329472, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b130a6de000
mmap(NULL, 4591616, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b130aaff000
mmap(NULL, 4853760, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b130af60000
mmap(NULL, 5115904, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b130b401000
mmap(NULL, 5378048, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b130b8e2000
mmap(NULL, 5640192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b130be03000
mmap(NULL, 5902336, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b130c364000
mmap(NULL, 6164480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b130c905000
mmap(NULL, 6426624, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b130cee6000
...
---------------------------------------------

So, I looked into that who allocated that many pages.
(I reproduced the situation for debugging, and some addresses are different as before.)

---------------------------------------------
...
Catchpoint 1 (call to syscall 'mmap'), 0x00002b8e14b8b0ea in mmap64 () from /lib64/libc.so.6
(gdb) bt
#0  0x00002b8e14b8b0ea in mmap64 () from /lib64/libc.so.6
#1  0x00002b8e14b2d9b6 in _int_malloc () from /lib64/libc.so.6
#2  0x00002b8e14b2e6df in _int_realloc () from /lib64/libc.so.6
#3  0x00002b8e14b2f3e2 in realloc () from /lib64/libc.so.6
#4  0x00002b8e1d75ab50 in zend_mm_mem_malloc_realloc (storage=0x2b8e1a762300, ptr=0x2b8e25399030, size=524288) at /root/php-5.4.39/Zend/zend_alloc.c:292
#5  0x00002b8e1d75efd9 in _zend_mm_realloc_int (heap=0x2b8e1a762320, p=0x2b8e25399090, size=261984, __zend_filename=0x2b8e1d8b5428 "/root/php-5.4.39/ext/json/json.c", __zend_lineno=530, __zend_orig_filename=0x0, __zend_orig_lineno
    at /root/php-5.4.39/Zend/zend_alloc.c:2292
#6  0x00002b8e1d75f7d5 in _erealloc (ptr=0x2b8e25399090, size=261984, allow_failure=0, __zend_filename=0x2b8e1d8b5428 "/root/php-5.4.39/ext/json/json.c", __zend_lineno=530, __zend_orig_filename=0x0, __zend_orig_lineno=0)
    at /root/php-5.4.39/Zend/zend_alloc.c:2446
#7  0x00002b8e1d5e65d5 in json_escape_string (buf=0x7fff42111ca0,
    s=0x2b8e1f9c7610 "abcdefg123456한글입니다abcdefg123456한글입니다abcdefg123456한글입니다abcdefg123456한글입니다abcdefg123456한글입니다abcdefg123456한글입니다abcdefg123456한글입니다abcd"..., len=1800,
    options=0) at /root/php-5.4.39/ext/json/json.c:530
#8  0x00002b8e1d5e25ba in json_encode_array (buf=0x7fff42111ca0, val=0x7fff421114a0, options=0) at /root/php-5.4.39/ext/json/json.c:299
#9  0x00002b8e1d5e8594 in php_json_encode (buf=0x7fff42111ca0, val=0x2b8e1f9c26e0, options=0) at /root/php-5.4.39/ext/json/json.c:642
#10 0x00002b8e1d5e238f in json_encode_array (buf=0x7fff42111ca0, val=0x7fff42111820, options=0) at /root/php-5.4.39/ext/json/json.c:279
#11 0x00002b8e1d5e8594 in php_json_encode (buf=0x7fff42111ca0, val=0x2b8e1f497b60, options=0) at /root/php-5.4.39/ext/json/json.c:642
#12 0x00002b8e1d5e275d in json_encode_array (buf=0x7fff42111ca0, val=0x7fff42111ba0, options=0) at /root/php-5.4.39/ext/json/json.c:304
#13 0x00002b8e1d5e8594 in php_json_encode (buf=0x7fff42111ca0, val=0x2b8e1f493ed8, options=0) at /root/php-5.4.39/ext/json/json.c:642
#14 0x00002b8e1d5e8d15 in zif_json_encode (ht=1, return_value=0x2b8e1f492978, return_value_ptr=0x0, this_ptr=0x0, return_value_used=1) at /root/php-5.4.39/ext/json/json.c:778
...
---------------------------------------------

json_encode() called so many mmap() syscalls and didn't call any munmap() syscalls.
Is this a memory leak of json_encode()? Or any other purpose like memory reuse?

Test script:
---------------
<?php
  // some large data from remote server consisting of utf8 unicde char.
  // (It's a just example.)
  $text = str_repeat("abcdefg123456한글입니다", 100);
  $data = "{\"result\":[" . str_repeat("{\"" . $text . "\":\"" . $text . "\"},", 1000);
  $data = rtrim($data, ",");
  $data .= "]}";

  // make a json object
  $json = json_decode($data);

  // manipulating the json object

  // return a result
  echo json_encode($json);
?>

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports

[2015-03-27 04:17 UTC] zoopi01 at gmail dot com

I've found some code blocks look like reuse memory; zend_alloc.c:zend_mm_search_large_block()

It seems that zend_mm_saerch_large_block() will provide large memory blocks for the caller
with no allocation overhead while it remains a memory pool(I suppose that is heap->large_free_buckets).

But something is strange.

This is pmap/smaps output.
---------------------------------------------
# pmap -x <pid>
...
00002b8785956000    9400    8504    8504 rw---    [ anon ]
00002b8786461000   28696   25820   25820 rw---    [ anon ]
00002b87883bd000    2756    2416    2416 rw---    [ anon ]
...
00002b878c6ff000  142680  139840  139840 rw---    [ anon ]
00007fff062f0000      84      48      48 rw---    [ stack ]
00007fff063fd000      12       4       0 r-x--    [ anon ]
...


# cat /proc/<pid>/smaps
...
2b8786461000-2b8788067000 rw-p 2b8786461000 00:00 0
Size:             28696 kB
Rss:              25820 kB
Shared_Clean:         0 kB
Shared_Dirty:         0 kB
Private_Clean:        0 kB
Private_Dirty:    25820 kB
Swap:                 0 kB
Pss:              25820 kB
...
2b878c6ff000-2b8795255000 rw-p 2b878c6ff000 00:00 0
Size:            142680 kB
Rss:             139840 kB
Shared_Clean:         0 kB
Shared_Dirty:         0 kB
Private_Clean:        0 kB
Private_Dirty:   139840 kB
Swap:                 0 kB
Pss:             139840 kB
...
---------------------------------------------

The largest anon pages were allocated in the previous request.

and gdb output.
---------------------------------------------
(in the previous request)
...
Breakpoint 3, 0x00002b877add0870 in free () from /lib64/libc.so.6
(gdb) bt
#0  0x00002b877add0870 in free () from /lib64/libc.so.6
#1  0x00002b8783892b6b in zend_mm_mem_malloc_free (storage=0x2b87885132c0, ptr=0x2b87936b2010) at /root/php-5.4.39/Zend/zend_alloc.c:297
#2  0x00002b8783892bed in zend_mm_del_segment (heap=0x2b87885132e0, segment=0x2b87936b2010) at /root/php-5.4.39/Zend/zend_alloc.c:924
#3  0x00002b87838962bc in _zend_mm_free_int (heap=0x2b87885132e0, p=0x2b87936b2070, __zend_filename=0x2b87839ed428 "/root/php-5.4.39/ext/json/json.c", __zend_lineno=782, __zend_orig_filename=0x0, __zend_orig_lineno=0)
    at /root/php-5.4.39/Zend/zend_alloc.c:2111
#4  0x00002b878389774c in _efree (ptr=0x2b87936b2070, __zend_filename=0x2b87839ed428 "/root/php-5.4.39/ext/json/json.c", __zend_lineno=782, __zend_orig_filename=0x0, __zend_orig_lineno=0)
    at /root/php-5.4.39/Zend/zend_alloc.c:2436
#5  0x00002b8783720da3 in zif_json_encode (ht=1, return_value=0x2b87855ca978, return_value_ptr=0x0, this_ptr=0x0, return_value_used=1) at /root/php-5.4.39/ext/json/json.c:782
#6  0x00002b878390a443 in zend_do_fcall_common_helper_SPEC (execute_data=0x2b87855970e8) at /root/php-5.4.39/Zend/zend_vm_execute.h:643
#7  0x00002b878390f83f in ZEND_DO_FCALL_SPEC_CONST_HANDLER (execute_data=0x2b87855970e8) at /root/php-5.4.39/Zend/zend_vm_execute.h:2233
#8  0x00002b8783909561 in execute (op_array=0x2b87855ca7b8) at /root/php-5.4.39/Zend/zend_vm_execute.h:410
#9  0x00002b87838cf50f in zend_execute_scripts (type=8, retval=0x0, file_count=3) at /root/php-5.4.39/Zend/zend.c:1329
#10 0x00002b878383ade7 in php_execute_script (primary_file=0x7fff06302930) at /root/php-5.4.39/main/main.c:2502
#11 0x00002b8783975407 in php_handler (r=0x2b877934b128) at /root/php-5.4.39/sapi/apache2handler/sapi_apache2.c:667
#12 0x00002b877929c92a in ap_run_handler ()
#13 0x00002b877929fdb2 in ap_invoke_handler ()
#14 0x00002b87792aad78 in ap_process_request ()
#15 0x00002b87792a7e20 in ?? ()
#16 0x00002b87792a3e62 in ap_run_process_connection ()
#17 0x00002b87792af54d in ?? ()
#18 0x00002b87792af7fa in ?? ()
#19 0x00002b87792b005d in ap_mpm_run ()
#20 0x00002b87792896e0 in main ()
...

(in the current request)
...
Breakpoint 4, zend_mm_search_large_block (heap=0x2b87885132e0, true_size=5688) at /root/php-5.4.39/Zend/zend_alloc.c:1821
1821                    p = heap->large_free_buckets[index];
(gdb) bt
#0  zend_mm_search_large_block (heap=0x2b87885132e0, true_size=5688) at /root/php-5.4.39/Zend/zend_alloc.c:1821
#1  0x00002b87838957c2 in _zend_mm_alloc_int (heap=0x2b87885132e0, size=5600, __zend_filename=0x2b87839ed428 "/root/php-5.4.39/ext/json/json.c", __zend_lineno=423,
    __zend_orig_filename=0x2b8783a21d68 "/root/php-5.4.39/Zend/zend_alloc.c", __zend_orig_lineno=2539) at /root/php-5.4.39/Zend/zend_alloc.c:1934
#2  0x00002b87838976d1 in _emalloc (size=5600, __zend_filename=0x2b87839ed428 "/root/php-5.4.39/ext/json/json.c", __zend_lineno=423, __zend_orig_filename=0x2b8783a21d68 "/root/php-5.4.39/Zend/zend_alloc.c",
    __zend_orig_lineno=2539) at /root/php-5.4.39/Zend/zend_alloc.c:2425
#3  0x00002b878389789e in _safe_emalloc (nmemb=2800, size=2, offset=0, __zend_filename=0x2b87839ed428 "/root/php-5.4.39/ext/json/json.c", __zend_lineno=423, __zend_orig_filename=0x0, __zend_orig_lineno=0)
    at /root/php-5.4.39/Zend/zend_alloc.c:2539
#4  0x00002b878371c2b3 in json_escape_string (buf=0x7fff063001d0,
    s=0x2b87855d0ab8 "abcdefg123456한글입니다abcdefg123456한글입니다abcdefg123456한글입니다abcdefg123456한글입니다abcdefg123456한글입니다abcdefg123456한글입니다abcdefg123456한글입니다abcd"...,
    len=2800, options=0) at /root/php-5.4.39/ext/json/json.c:423
#5  0x00002b878371a5ba in json_encode_array (buf=0x7fff063001d0, val=0x7fff062ff9d0, options=0) at /root/php-5.4.39/ext/json/json.c:299
#6  0x00002b8783720594 in php_json_encode (buf=0x7fff063001d0, val=0x2b87855ccdd8, options=0) at /root/php-5.4.39/ext/json/json.c:642
#7  0x00002b878371a38f in json_encode_array (buf=0x7fff063001d0, val=0x7fff062ffd50, options=0) at /root/php-5.4.39/ext/json/json.c:279
#8  0x00002b8783720594 in php_json_encode (buf=0x7fff063001d0, val=0x2b87855cccc0, options=0) at /root/php-5.4.39/ext/json/json.c:642
#9  0x00002b878371a75d in json_encode_array (buf=0x7fff063001d0, val=0x7fff063000d0, options=0) at /root/php-5.4.39/ext/json/json.c:304
#10 0x00002b8783720594 in php_json_encode (buf=0x7fff063001d0, val=0x2b87855cd170, options=0) at /root/php-5.4.39/ext/json/json.c:642
#11 0x00002b8783720d15 in zif_json_encode (ht=1, return_value=0x2b87855cab88, return_value_ptr=0x0, this_ptr=0x0, return_value_used=1) at /root/php-5.4.39/ext/json/json.c:778
#12 0x00002b878390a443 in zend_do_fcall_common_helper_SPEC (execute_data=0x2b87855970e8) at /root/php-5.4.39/Zend/zend_vm_execute.h:643
#13 0x00002b878390f83f in ZEND_DO_FCALL_SPEC_CONST_HANDLER (execute_data=0x2b87855970e8) at /root/php-5.4.39/Zend/zend_vm_execute.h:2233
#14 0x00002b8783909561 in execute (op_array=0x2b87855ca968) at /root/php-5.4.39/Zend/zend_vm_execute.h:410
#15 0x00002b87838cf50f in zend_execute_scripts (type=8, retval=0x0, file_count=3) at /root/php-5.4.39/Zend/zend.c:1329
#16 0x00002b878383ade7 in php_execute_script (primary_file=0x7fff06302930) at /root/php-5.4.39/main/main.c:2502
#17 0x00002b8783975407 in php_handler (r=0x2b877934d138) at /root/php-5.4.39/sapi/apache2handler/sapi_apache2.c:667
#18 0x00002b877929c92a in ap_run_handler ()
#19 0x00002b877929fdb2 in ap_invoke_handler ()
#20 0x00002b87792aad78 in ap_process_request ()
#21 0x00002b87792a7e20 in ?? ()
#22 0x00002b87792a3e62 in ap_run_process_connection ()
#23 0x00002b87792af54d in ?? ()
#24 0x00002b87792af7fa in ?? ()
#25 0x00002b87792b005d in ap_mpm_run ()
#26 0x00002b87792896e0 in main ()
(gdb) print heap->large_free_buckets
$1 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x2b8786a63cb8, 0x2b8786a6aac8, 0x2b87855cda90, 0x0, 0x0, 0x0, 0x2b8786a6e2a8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}
---------------------------------------------

There doesn't exists the largest anon pages in heap->large_free_buckets.
But that pages are still allocated in the same size.

[2017-01-09 19:08 UTC] bukka@php.net

-Status: Open +Status: Feedback -Assigned To: +Assigned To: bukka

[2017-01-09 19:08 UTC] bukka@php.net

It can't be fixed on any version of PHP 5 as only 5.6 is open for security fixes. Do you see anything like this on PHP 7.1?

[2017-01-22 04:22 UTC] php-bugs at lists dot php dot net

No feedback was provided. The bug is being suspended because
we assume that you are no longer experiencing the problem.
If this is not the case and you are able to provide the
information that was requested earlier, please do so and
change the status of the bug back to "Re-Opened". Thank you.

[2018-11-26 21:00 UTC] contact at fernandocarletti dot net

We are having this issue too. It's quite easy to reproduce, as noted in this link: http://bestphptrainingtrivandrum.blogspot.com/2017/09/memory-leak-in-7x-jsondecode-best-php.html

Here's the snippet that can reproduce it: 

<?php
echo memory_get_usage(false) . ' : ' .memory_get_usage(true) . PHP_EOL;

$json = json_decode(file_get_contents('http://zaremedia.com/big.json'));
echo memory_get_usage(false) . ' : ' .memory_get_usage(true) . PHP_EOL;

unset($json);
echo memory_get_usage(false) . ' : ' .memory_get_usage(true) . PHP_EOL;

[2018-11-26 21:08 UTC] nikic@php.net

@contact at fernandocarletti dot net: You can generally ignore increased usage reported by memory_get_usage(true). If you really want to reclaim that memory immediately, run gc_mem_caches(). Generally there should be no need to do so though.

	php.net \| support \| documentation \| report a bug \| advanced search \| search howto \| statistics \| random bug \| login
go to bug id or search bugs for


Copyright © 2001-2024 The PHP Group All rights reserved.	Last updated: Sun Dec 22 06:01:30 2024 UTC