Matt, unfortunately I can't reproduce the fails on my laptops, they're probably too slow. Nevertheless I'm not sure the patch does expected things. I've just debugged on win7 and ubuntu - SO_REUSEADDR is defined everywhere. MSDN even says it's available on all the versions. This means setsockopt should work as expected.

Besides that, removing that wrapping #ifdef could have impacts in some rare cases, i think, as SO_REUSEADDR is marked as a BSD macro, so despite it's luckily present on GNU, it might be absent on some POSIX-complaint only systems. Could you please explain why this fix is correct?

Also the php_cli_server_017.phpt was still failing for me, but that's just the unix/windows dir separator.

just came into my mind - may be it would have an effect if we just set a timeout at the top of the  php_cli_server_start() too? so the previous server had a chance to exit ...

The test has been fixed in Git.

Setting SO_REUSEADDR socket option will not only make sure all the tests pass every time, but also if user restarts the web server, it will make sure the web server can listen on that port when it restarts.

Most TCP servers set SO_REUSEADDR option.

This is a real problem on Windows and Linux and probably *BSD. Fixing it for those platforms is worth causing a minor compile problem for some weird os that doesn't have SO_REUSEADDR.

Alternatively, adding a sleep() call to the test after the web server exits should probably work, but it would make the test very slow (might have to wait 60 seconds for each of 17 tests => 17 minutes).

Matt, what i mean SO_REUSEADDR is defined on any Windows, see http://msdn.microsoft.com/en-us/library/windows/desktop/ms740621(v=vs.85).aspx , as well as any other popular platform like gnu/bsd/solaris. And if it's defined, the piece of code with setsockopt is compiled and the option is set disregarding it's compiled with or without the wrapping #ifdef. Otherwise, if it were not defined, there were a compile error when you remove the wrapping #ifdef. That's why i wonder it changes something, because there is really no difference.

<?php

// this script executes all 17 php_cli_server*phpt tests using run-tests.php
// 500 times and counts up the test failures

$php = "c:\php-sdk\php-src-PHP-5.4-31ab04e\Release_TS\php.exe";



// note: script assumes run from within test-pack root

$fh = fopen("cli_server_tests.txt", 'w');
fwrite($fh, "Sapi/cli/tests/php_cli_server_001.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_002.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_003.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_004.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_005.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_006.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_007.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_008.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_009.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_010.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_011.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_012.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_013.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_014.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_015.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_016.phpt".PHP_EOL);
fwrite($fh, "Sapi/cli/tests/php_cli_server_017.phpt".PHP_EOL);
fclose($fh);

function count_run_test_failures() {
	global $php;
	
	$lines = array();
	exec("set TEST_PHP_EXECUTABLE=$php & $php run-test.php -r cli_server_tests.txt & exit", $lines);
	
	// parse $lines for
	$failure_count = 0;
	foreach ($lines as $line) {
		// "Tests failed    :    3 ( 17.6%) ( 17.6%)"
		preg_match("/Tests failed    : .+ (\d)+ .+/", $line, $m);
		
		if (count($m)==2) {
			$failure_count = $m[1] + 0; // to int
			break;
		}
	}
	//sleep(50);
	
	return $failure_count;
}

//var_dump(count_run_test_failures());
//exit;

$run_count = 0;
$total_failures = 0;
while ( $run_count < 500 ) {
	$failure_count = count_run_test_failures();
	$total_failures += $failure_count;
	
	if ($failure_count != 0) {
		echo "$failure_count fails $total_failures total" . PHP_EOL;
	} else {
		echo "pass $run_count run" . PHP_EOL;
	}

	$run_count++;
}

$avg_fails = $total_failures / $run_count;

echo "Total Failures $total_failures Runs $run_count Avg fails $avg_fails" . PHP_EOL;
echo " * 17 individual phpts for each run " . PHP_EOL;

?>

My newest patch fixes the remaining problems with these tests on Windows.

Using the attached script, I've run all the php_cli_server*phpt tests on Windows and Gentoo Linux 500 times without any failure (using php-5.4 r31ab04e).

This bug has been fixed in SVN.

Snapshots of the sources are packaged every three hours; this change
will be in the next snapshot. You can grab the snapshot at
http://snaps.php.net/.

 For Windows:

http://windows.php.net/snapshots/
 
Thank you for the report, and for helping us make PHP better.