php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #71355 Fail to use huge page on power8 because of wrong page size
Submitted: 2016-01-12 23:06 UTC Modified: 2016-03-26 09:04 UTC
Votes:5
Avg. Score:4.6 ± 0.8
Reproduced:5 of 5 (100.0%)
Same Version:4 (80.0%)
Same OS:4 (80.0%)
From: jocelyn dot fournier at softizy dot com Assigned:
Status: Open Package: opcache
PHP Version: 7.0.2 OS: Linux on Power
Private report: No CVE-ID: None
View Developer Edit
 [2016-01-12 23:06 UTC] jocelyn dot fournier at softizy dot com 
Description:
------------
Hi,

It seems huge page size is hardcoded in the php src code : 

In ZendAccelerator.c for example, you have :

static void accel_move_code_to_huge_pages(void)
{
	FILE *f;
	long unsigned int huge_page_size = 2 * 1024 * 1024;

However on IBM Power8 on linux, the default page size is 64KB ( http://www.redbooks.ibm.com/redbooks/pdfs/sg248171.pdf ).
Hence when huge pages are enabled, php displays the following warning : 


munmap() failed: [22] Invalid argument

strace output :

[...]
mmap(NULL, 2097152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|0x40000, -1, 0) = 0x3effff000000
madvise(0x3effff000000, 2097152, MADV_HUGEPAGE) = -1 EINVAL (Invalid argument)
[...]
open("/usr/local/lib/php/extensions/no-debug-non-zts-20151012/apcu.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\25\0\1\0\0\0\0`\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=121864, ...}) = 0
mmap(NULL, 158952, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3fff8e140000
mmap(0x3fff8e160000, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x10000) = 0x3fff8e160000
close(3)                                = 0
[...]
munmap(0x3efff7000000, 134217728)       = 0
close(3)                                = 0
munmap(0x3fff962c0000, 185272)          = 0
munmap(0x3fff96240000, 82352)           = 0
munmap(0x3fff96260000, 328760)          = 0
munmap(0x3fff94210000, 33554432)        = 0
munmap(0x3fff962f0000, 158952)          = 0
brk(0x1001a8d0000)                      = 0x1001a8d0000
munmap(0x3fff96320000, 301576)          = 0
munmap(0x3fff96210000, 196608)          = 0
munmap(0x3effff000000, 2097152)         = -1 EINVAL (Invalid argument)
write(2, "\nmunmap() failed: [22] Invalid a"..., 40
munmap() failed: [22] Invalid argument
) = 40
munmap(0x3fff96370000, 327680)          = 0
exit_group(0)                           = ?
+++ exited with 0 +++


To fix the issue, the default page size should not be hardcoded, but should depend on the platform.

Thanks and regards,
  Jocelyn Fournier


Expected result:
----------------
No warning

Actual result:
--------------
munmap() failed: [22] Invalid argument

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2016-02-16 21:57 UTC] basu at us dot ibm dot com 
Since linux on POWER supports huge page of size 16MB, it appears that this
hard coded value of 2MB in ZendAccelerator.c is letting the mmap()/madvise()/munmap()
fail with EINVAL due to invalid argument.

This routine below with 2MB constant is only valid for promoting opcode cache to huge page.. This routine is not controlling the huge page semantics for PHP data allocations. However, there is a CHUNKSIZE buckets that are being managed by PHP 7.0.2
that does not have a size for huge page (max size for chunk size is only 2MB) I changed that to 16MB and was able to eliminate the mmap/madvise/munmap errors. Basically 
the chunk size management code should include huge page support on POWER.
 [2016-03-21 17:47 UTC] jpauli@php.net 
The problem is that there is no way do dynamically discover the huge page size, knowing that some systems have different sizes, and may map different memory arear using different page size. There is nothing comparable to _SC_PAGESIZE or getpagesize(), which are part of POSIX.

One could parse /proc/mounts to find hugetlbfs entries.
Or we could rely on the excellent libhugetlbfs, but what if the system doesn't support it ?

https://github.com/libhugetlbfs/libhugetlbfs

Huge page mapping is really hard to manage in a crossplatform way, way more than classical memory mappings where we have libc's malloc() on top of the Kernel calls, and where (nearly) every system agreed on having a default 4Kb page size
 [2016-03-21 19:47 UTC] jocelyn dot fournier at softizy dot com 
/proc/meminfo should give you the Hugepagesize info.
What about using HPAGE_SIZE ?
 [2016-03-22 11:56 UTC] jpauli@php.net 
I wasn't aware of such a define.

We could use it if we can detect it at compile time, yes.
Could you try on your system if that gives good results ?
 [2016-03-22 16:25 UTC] basu at us dot ibm dot com 
Hi,

All ppc64 architectures (POWER) support 16MB page support at the hardware level. Also,
the Linux kernel supports 16MB pages if it is built with CONFIG_HUGETLBFS (and CONFIG_HUGETLB_PAGE is also enabled) enabled. As long as MAP_HUGETLB is used in mmap()
calls from applications, (No need for mount hugetlbfs) and lenth parameter is 
huge page size aligned, the calls mmap() and munmap() will be successful.

Now, there are two ways PHP code can be sure of whether system supports hugepages.
1) It can look at /proc/meminfo:   cat /proc/meminfo | grep Hugepagesize to get 
the current pagesize for use with mmap()/munmap() if MAP_HUGETLB is to be used.
2) we could place the policy on the user, i.e., have the user specify in php conf
file the intention for php to use huge pages during mmap()/munmap(). For example,
if we specify USE_HUGEPAGES=1 in php conf file, php could use this at run time 
as a hint to start using hugepages. Of course, the user would set the sysctl.conf file to have vm.nr_hugepages set.
 [2016-03-26 09:04 UTC] rasmus@php.net 
I think option 2 makes more sense. We have now disabled huge pages by default as of PHP 7.0.6 and there is an env var called USE_ZEND_ALLOC_HUGE_PAGES that enables it. That is currently a boolean, but we could change it to take the page size.
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved. 		Last updated: Thu Nov 21 14:01:29 2024 UTC