php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Request #29571 [PATCH] Many uploads will slow down Linux machine
Submitted: 2004-08-08 14:42 UTC Modified: 2015-01-08 23:08 UTC
From: k at ailis dot de Assigned:
Status: Open Package: *General Issues
PHP Version: * OS: Linux
Private report: No CVE-ID: None
View Add Comment Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
You can add a comment by following this link or if you reported this bug, you can edit this bug over here.
(description)
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: k at ailis dot de
New email:
PHP Version: OS:

 

 [2004-08-08 14:42 UTC] k at ailis dot de
Description:
------------
This is a mix of "performance problem" and "feature request". So feel free to reclassify this report as "Feature request" if you think it fits better.

In a  large project we have the problem that many users are uploading many large files to our PHP server. PHP can handle this very well but the problem is, that all files are stored in the same directory. Our upload directory sometimes contains more than 20000 files. This produces high loads because accessing these files is very I/O intensive because of the flat directory hierarchy. 

You already have addressed this problem with session files by introducing a "directory depth". Its possible to use "2;/var/lib/php/uploads" as session.save_path which is a very nice feature. Exactly this feature would be a cool thing if it were also implemented for "upload_tmp_dir".

Here is a small patch as an example how this feature could be implemented. We are using this patch on our servers and it works great. This patch not only implements this feature for upload_tmp_dir but for all other modules which are using the php_open_temporary_file() function.

--- php_open_temporary_file.c.orig	Sun Aug  8 14:29:06 2004
+++ php_open_temporary_file.c	Sun Aug  8 14:29:12 2004
@@ -106,6 +106,10 @@
 {
 	char *trailing_slash;
 	char *opened_path;
+	char *p;
+	char *sub_dirs;
+	size_t dirdepth, i;
+	int rand_value;
 	int fd = -1;
 #ifndef HAVE_MKSTEMP
 	int open_flags = O_CREAT | O_TRUNC | O_RDWR
@@ -125,6 +129,30 @@
 	if (!(opened_path = emalloc(MAXPATHLEN))) {
 		return -1;
 	}
+	
+	/* Check subdirectory depth and point p to real path name */
+	if ((p = strchr(path, ';')))
+	{
+	    dirdepth = (size_t) strtol(path, NULL, 10);
+	    p++;
+	}
+	else
+	{
+	    dirdepth = 0;
+	    p = (char *) path;
+	}
+    
+	/* Build subdirectory tree if needed */
+	if (dirdepth) {
+	    if (!(sub_dirs = emalloc(dirdepth * 2 + 1))) return -1;
+	    sub_dirs[0] = 0;
+	    for (i = 0; i < dirdepth; i++)
+	    {
+	        strcat(sub_dirs, "x/");
+	        rand_value = php_rand() & 0xf;
+	        sub_dirs[i * 2] = rand_value < 10 ? rand_value + 48 : rand_value + 87;
+	    }
+	} else sub_dirs = "";
 
 	if (IS_SLASH(path[strlen(path)-1])) {
 		trailing_slash = "";
@@ -132,7 +160,10 @@
 		trailing_slash = "/";
 	}
 
-	(void)snprintf(opened_path, MAXPATHLEN, "%s%s%sXXXXXX", path, trailing_slash, pfx);
+	(void)snprintf(opened_path, MAXPATHLEN, "%s%s%s%sXXXXXX", p, trailing_slash, sub_dirs, pfx);
+	
+	/* Free memory if it was used */
+	if (dirdepth) efree(sub_dirs);
 
 #ifdef PHP_WIN32
 	if (GetTempFileName(path, pfx, 0, opened_path)) {
@@ -140,7 +171,7 @@
 		 * which means that opening it will fail... */
 		VCWD_CHMOD(opened_path, 0600);
 		fd = VCWD_OPEN_MODE(opened_path, open_flags, 0600);
-	}
+l1	}
 #elif defined(NETWARE)
 	/* Using standard mktemp() implementation for NetWare */
 	file_path = mktemp(opened_path);



Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2015-01-08 23:08 UTC] ajf@php.net
-Package: Feature/Change Request +Package: *General Issues -PHP Version: 4.3.8 +PHP Version: *
 [2015-01-08 23:08 UTC] ajf@php.net
Seems reasonable.
 
PHP Copyright © 2001-2017 The PHP Group
All rights reserved.
Last updated: Sun Nov 19 01:31:42 2017 UTC