php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #81304 simple socket listen and receive scripts transfer incomplete data
Submitted: 2021-07-28 00:48 UTC Modified: 2022-05-30 07:26 UTC
From: odarcan at gmail dot com Assigned:
Status: Not a bug Package: Sockets related
PHP Version: 7.4.21 OS: linux
Private report: No CVE-ID: None
 [2021-07-28 00:48 UTC] odarcan at gmail dot com
Description:
------------
a simple tcp socket listen loop (1listen.php) that sends 12mb data to whoever connects to port 1234 and writes "cmd"

the receiver script connects to localhost:1234 and writes "cmd" and counts the number of bytes received (2receive.php) receives different amount and rarely, but sometimes the correct amount.

1listen.php

<?php
ini_set('error_reporting', E_ALL ^ E_NOTICE);
ini_set('display_errors', 1);

set_time_limit (0);
$address = '0.0.0.0';
$port = 1234;
function o($t,$x="\n"){ //just placebo for echo
	echo "s_l: $t$x";
}

while (1){
	$linger = array ('l_linger' => 0, 'l_onoff' => 1);
	$socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
 	socket_set_option($socket, SOL_SOCKET, SO_LINGER, $linger);

	$r=socket_bind($socket, $address, $port);
	
	if ($r==false){
		var_dump($socket);
		o("Could not bind to address: $address $port");
		o("sleeping...");
		unset($socket);
		sleep(1);
		continue;
	}
	$r=socket_listen($socket);
	
	if ($r==false){
		o("could not listen to socket on $port");
		sleep(5);
		continue;
	}

	o("Listening on $address:$port...");
	$c=0;

	while (true){
		$msgsocket = socket_accept($socket);
		socket_set_option($msgsocket, SOL_SOCKET, SO_LINGER, $linger);
		$received= socket_read($msgsocket,2048,PHP_BINARY_READ);
		
		if($received ===false){
			 echo "socket read error: ".socket_strerror(socket_last_error($msgsock)) . "\n";
			 sleep(4);
			 continue 2;
		}
		
		o("-----------------------");
		
		if ($received !==""){
			$string=trim($received);
			echo "received: [$string]\n";
			$cmd=explode(" ",$string);
		
		
			if ($cmd[0]=="cmd"){
				$domain=$cmd[1];
				$md5=$cmd[2];
				
				$arr=[]; 
				$n=1;
				for($a=0;$a<1000;$a++){
				for($b=0;$b<1000;$b++){
					$arr[$a][$b]=$a;//generate data to send
				}
				}
				
				
				$r2=serialize($arr);
				$l1=strlen($r2);
				$l2=socket_write($msgsocket,$r2);
				
				if ($l2<$l1){
					o("incomplete transfer");
				}
				o("wrote $l1 / $l2 bytes");
					if ($l2===false){
					o("error failure");
				}
			}
		}
		socket_close($msgsocket);
	}
	socket_close($socket);
}

shutdown();
?>



2receive.php

<?php
$m="cmd";
$r=sendclosetcp("localhost",1234,$m);

echo strlen($r);
echo "\n";
function o($t,$x="\n"){
	global $domain;
	$computer="";
	echo $computer." $t$x";
}

function sendclosetcp($ip,$port,$msg){
	$clnt_sock = socket_create(AF_INET, SOCK_STREAM, getprotobyname('tcp'));
	$r=socket_connect($clnt_sock, $ip, $port);
	
	if ($r==false){
		o("could not connect to $ip $port");
		return "";
	}
	socket_write($clnt_sock,$msg);
	
	$ret="";
	while(1){
		$buf=@socket_read($clnt_sock, 10000, PHP_BINARY_READ);
		if (strlen($buf)<19) 
		var_dump($buf);
		$ret.=$buf;
		$e=socket_last_error($clnt_sock);
		
		if ($buf===false){
			o("1 remote host closed the connection $e");
			break;
		}
		if ($buf==false){
			o("2 remote host closed the connection $e");
			break;
		}	
		if ($buf==""){
			o("3 remote host closed the connection $e");
			break;
		}	
		if ($buf===""){
			o("4 remote host closed the connection $e");
			break;
		}
	}
	
	o("received ".strlen($ret)." bytes");
	return $ret;
}
?>

Test script:
---------------
run 1listen.php in one terminal window
and 2receive.php in another

each time you run 2receive.php it receives different amount of bytes before the connection is closed, I have tried on more than one computer


Expected result:
----------------
"sent x bytes"
and 
"received x bytes" 

should be the same number

seemingly randomly for files over a few mbs, the received amount varies and data transfers are incomplete

Actual result:
--------------
1listen.php: wrote 11794899 / 11794899 bytes

for example
2receive.php:

received 11053512 bytes
received 8039296 bytes
received 9108280 bytes

etc.

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2021-07-28 01:00 UTC] odarcan at gmail dot com
-Summary: simple socket listen and send scripts transfer incomplete data +Summary: simple socket listen and receive scripts transfer incomplete data
 [2021-07-28 01:00 UTC] odarcan at gmail dot com
you can add 

if (unserialize($r)===false){
 echo "incomplete data transfer\n";
}

to the line under $r=sendclosetcp(....
to warn that the data sent is incomplete.
 [2021-07-28 01:14 UTC] requinix@php.net
-Status: Open +Status: Not a bug
 [2021-07-28 01:14 UTC] requinix@php.net
Sorry, but your problem does not imply a bug in PHP itself.  For a
list of more appropriate places to ask for help using PHP, please
visit http://www.php.net/support.php as this bug system is not the
appropriate forum for asking support questions.  Due to the volume
of reports we can not explain in detail here why your report is not
a bug.  The support channels will be able to provide an explanation
for you.

Thank you for your interest in PHP.


 [2021-07-29 00:58 UTC] odarcan at gmail dot com
-Type: Bug +Type: Documentation Problem
 [2021-07-29 00:58 UTC] odarcan at gmail dot com
solution is this apparently socket_write causes buffer overrun when the msg is too big, so send it in chunks instead

$l1=strlen($msg);
$totaltransferred=0;

$bsize=10000;
for ($a=0;$a<$l1;$a+=$bsize){
	$part=substr($msg,$a,$bsize);
	$l2=socket_write($msgsocket,$part,strlen($part));
	$totaltransferred+=$l2;
}
	
if ($totaltransferred<$l1){
	o("incomplete transfer");
}
 [2021-07-29 01:22 UTC] requinix@php.net
-Type: Documentation Problem +Type: Bug
 [2021-07-29 01:22 UTC] requinix@php.net
Please note that the term "buffer overrun" means something very different from what you meant.

https://www.php.net/manual/en/function.socket-write.php
> Note:
> socket_write() does not necessarily write all bytes from the given buffer.
> It's valid that, depending on the network buffers etc., only a certain amount
> of data, even one byte, is written though your buffer is greater. You have to
> watch out so you don't unintentionally forget to transmit the rest of your
> data.
 [2022-05-29 12:44 UTC] odarcan at gmail dot com
I dont see how tcp can send incomplete data and this is accepted as normal,

I really how someone can explain or point me in the right direction,
nothing is mentioned about this in the php manuals or any inclination on how do replicate or overcome such a phenomenon or that it is something common, nor that it is even something commonly known.

that tcp randomly transfers less bytes goes against everything I have learned about tcp.

please explain or tell me what this phenomena is called and where in literature it can even be found.

if it is not a php issue or maybe a bug in some other system or the many pcs Ive tried it on that would be also acceptable

thanks I am awaiting a relevant reply.
 [2022-05-29 13:24 UTC] odarcan at gmail dot com
Also in a variant of this where the data is sent in a loop,

making the buffer size 10000 and adding a usleep(10000) to the loop SOLVES the problem

I dont see how this is not a php problem

something is writing somewhere too fast, whatever that is Id like to know what and how I can prevent it.
 [2022-05-29 20:34 UTC] requinix@php.net
The main problem here seems to be that you have some knowledge about networking but aren't aware that you're missing out on a lot more.

https://www.google.com/search?q=socket+read+write+buffering
 [2022-05-30 07:26 UTC] odarcan at gmail dot com
1. Have you tried my code?

2. Do you know how to change it to make it transfer the exact correct amount every time?

3. Do you find it peculiar that none of the pages for socket functions in php manual neither MENTION nor SHOW anything about additional precautions in order to make tcp transfer exact and correct amounts every time?

Do you assume that it "should be working" and therefore need no additional explanation?

if you cant provide me with a correct solution on how to transfer 15 mb of data in php, via socket create, bind, listen, accept, read and write, I request that you reopen this ticket as a bug.

If you can, Id want to learn why it has not been mentioned in the manuals nor on the web.

I have done my share of programming with udp, tcp blocking and non blocking.

TCP is supposed to do complete transfers regardless of some sleep delay being added.

Broken connections almost every single time is not the norm for this.
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Sat Dec 14 07:01:27 2024 UTC