|  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #79013 Content-Length missing when posting a curlFile with curl
Submitted: 2019-12-21 16:19 UTC Modified: 2020-02-14 10:55 UTC
Avg. Score:5.0 ± 0.0
Reproduced:3 of 3 (100.0%)
Same Version:3 (100.0%)
Same OS:0 (0.0%)
From: christian at klemmer dot io Assigned: cmb (profile)
Status: Re-Opened Package: cURL related
PHP Version: 7.4.1 OS: debian 10.2
Private report: No CVE-ID: None
Have you experienced this issue?
Rate the importance of this bug to you:

 [2019-12-21 16:19 UTC] christian at klemmer dot io
When posting a curlFile with curl, I get different results with the same code (and curlFile) if I run it with PHP 7.3 or PHP 7.4.

"Content-Length" is missing and instead replaced by "Transfer-Encoding: chunked". I query an API which is behind an haproxy server. Missing content-lenght seems to remove all files posted on their side.
Nevertheless I would've expected the same behaviour with PHP 7.4 running with the same curl-version.

I'm using the repository on a debian 10.2 server.
dpkg -l | grep curl
ii  curl                                 7.64.0-4
ii  libcurl3-gnutls:amd64                7.64.0-4
ii  libcurl4:amd64                       7.64.0-4
ii  php7.3-curl                          7.3.13-1+0~20191218.50+debian10~1.gbp23c2da
ii  php7.4-curl                          7.4.1-1+0~20191218.8+debian10~1.gbp21c50e
ii  python3-pycurl             

curl-information out of phpinfo (7.3 testenv and 7.4 testenv are exactly the same):

cURL support	enabled
cURL Information	7.64.0
Age	4
AsynchDNS	Yes
CharConv	No
Debug	No
GSS-Negotiate	No
IPv6	Yes
krb4	No
Largefile	Yes
libz	Yes
Protocols	dict, file, ftp, ftps, gopher, http, https, imap, imaps, ldap, ldaps, pop3, pop3s, rtmp, rtsp, scp, sftp, smb, smbs, smtp, smtps, telnet, tftp
Host	x86_64-pc-linux-gnu
SSL Version	OpenSSL/1.1.1d
ZLib Version	1.2.11
libSSH Version	libssh2/1.8.0

Test script:
$filename = 'testfile.txt';
if(file_exists($filename) && is_file($filename) && (int)filesize($filename) > 0) {
    try {
        $ch = curl_init('') or die('API down');
        curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
        curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        curl_setopt($ch, CURLOPT_HEADER, true);
        curl_setopt($ch, CURLINFO_HEADER_OUT, true);
        curl_setopt($ch, CURLOPT_VERBOSE, true);
        curl_setopt($ch, CURLOPT_POST, true);
        curl_setopt($ch, CURLOPT_POSTFIELDS, array('test' => new CurlFile($filename)));
        $curlInfo = curl_getinfo($ch);
        echo '<pre>'; print_r($curlInfo["request_header"]); echo '</pre>';
    } catch(Exception $e) {
        echo $e->getMessage();

Expected result:
With PHP 7.3.13:
Accept: */*
Content-Length: 245
Content-Type: multipart/form-data; boundary=------------------------c129897ae3ed733b

Actual result:
With PHP 7.4.1:
Accept: */*
Transfer-Encoding: chunked
Content-Type: multipart/form-data; boundary=------------------------ef5ab6d1e2455a47


Add a Patch

Pull Requests

Add a Pull Request


AllCommentsChangesGit/SVN commitsRelated reports
 [2019-12-21 20:46 UTC]
-Assigned To: +Assigned To: cmb
 [2019-12-21 20:46 UTC]
Note that there's nothing inherently wrong with this behavior. Honestly, I think the bug is that your proxy somehow isn't allowing the file upload when it comes chunked instead of in full. I suggest looking into that.

Anyway, looks like this change happened during request #77711
where using a CURLFile opts into cURL's support for streams, and it can't know the stream length so it has no choice but to go chunked.
 [2019-12-22 12:49 UTC] christian at klemmer dot io
Reading a bit more about it, I'm not sure if "Transfer-Encoding: chunked" is allowed, when using HTTP/2.
"HTTP/2 doesn't support HTTP 1.1's chunked transfer encoding mechanism, as it provides its own, more efficient, mechanisms for data streaming."
 [2019-12-22 20:39 UTC]
But are you actually using HTTP/2? Sure, your example says "HTTP/2" in it, but the headers themselves are HTTP/1.

Is your repro script the same as what you posted here?
 [2019-12-22 20:47 UTC] christian at klemmer dot io
Yes, it's 100% the same script.
After writing down my reposcript I actually posted a little file to to test if it's working there also. It is.

And is also showing with HTTP/2 in Google Chrome, so I think it's capable of doing so.
 [2019-12-22 21:15 UTC]
Thing is, this doesn't make any sense.

1. I can't imagine cURL will use HTTP/2 without an Upgrade or explicit foreknowledge about remote support for it
2. I would also expect it to use HTTP/2 only if the passed options supported it (at least until it's more common)
3. Like I said, the headers are completely wrong

I only really see two possibilities: either something in this bug report is incorrect, or cURL has some sort of massive bug(s) regarding HTTP/2 support that nobody seems to have noticed until now. No offense but from where I'm sitting, only one of those is believable.

As for reproducing,

On Ubuntu with libcurl 7.58 I get a chunked HTTP/1.1 request with PHP 7.4.1 and an unchunked HTTP/1.1 request with PHP 7.3.13, so if something changed in cURL it was between 7.58 and 7.64.
 [2019-12-22 21:21 UTC] bugreports at gmail dot com
a) Curl prefers HTTP/2 when the sevrer supports it
b) the server of the reporter support HTTP2 as he statet
c) that below is curl against a http2 proxy with a http1 backend

the origin sned all the headers in the style of "Cache-Control" and the proxy talking http2 transofrms them to lowercase

curl --head https://localhost/

HTTP/2 200
date: Sun, 22 Dec 2019 21:19:35 GMT
strict-transport-security: max-age=31536000
x-frame-options: SAMEORIGIN
etag: 786c872ae3730afa5623a43d8ee3e38b
cache-control: private
last-modified: Mon, 28 Nov 2016 16:55:29 GMT
vary: Accept-Encoding,User-Agent
content-type: text/html; charset=ISO-8859-1
age: 0
 [2019-12-22 21:38 UTC]
Ah, I didn't know HTTP/2 support was indicated in the TLS handshake. That explains some.

So I'm back to the headers being wrong. Maybe I just don't have the right environment to test this.
 [2019-12-22 22:14 UTC] christian at klemmer dot io
I did some additional testing with some VMs:

Ubuntu 19.10 - PHP 7.4.1 - CURL 7.65.3 (7.65.3-1ubuntu3):
Accept: */*
Transfer-Encoding: chunked
Content-Type: multipart/form-data; boundary=------------------------d498818044360b85

Debian 9.11 - PHP 7.4.1 - CURL 7.52.1 (7.52.1-5+deb9u9):
Accept: */*
Content-Length: 210
Expect: 100-continue
Content-Type: multipart/form-data; boundary=------------------------a4980090fbc4beae

Debian 10.2 - PHP 7.4.1 - CURL 7.64.0 (7.64.0-4) (same versions as my initial bug report, but different machine):
Accept: */*
Transfer-Encoding: chunked
Content-Type: multipart/form-data; boundary=------------------------f8e60d58d7f0df39

Here are the steps to reproduce my exact test environment:
Install Debian 10.2 (buster).
apt install apt-transport-https lsb-release ca-certificates
curl -fsSL | apt-key add -
sh -c 'echo "deb $(lsb_release -sc) main" >> /etc/apt/sources.list'
apt update
apt install php7.4-bcmath php7.4-bz2 php7.4-cgi php7.4-cli php7.4-common php7.4-curl php7.4-dba php7.4-fpm php7.4-gd php7.4-imagick php7.4-imap php7.4-intl php7.4-json php7.4-ldap php7.4-mbstring php7.4-mysql php7.4-readline php7.4-soap php7.4-xml php7.4-zip
 [2019-12-22 22:41 UTC] christian at klemmer dot io
And as addition tests with PHP 7.3.13 with different curl versions.
All result in the expected behaviour.

Ubuntu 19.10 - PHP 7.3.13 - CURL 7.65.3 (7.65.3-1ubuntu3):
Accept: */*
Content-Length: 210
Content-Type: multipart/form-data; boundary=------------------------4d274c175c3e77e5

Debian 9.11 - PHP 7.3.13 - CURL 7.52.1 (7.52.1-5+deb9u9):
Accept: */*
Content-Length: 210
Expect: 100-continue
Content-Type: multipart/form-data; boundary=------------------------5652b6f1c9e7a818

Debian 10.2 - PHP 7.3.13 - CURL 7.64.0 (7.64.0-4):
Accept: */*
Content-Length: 210
Content-Type: multipart/form-data; boundary=------------------------0b86b20895500f64

So it seems to be a combination of several CURL versions with PHP 7.4
 [2019-12-23 08:54 UTC]
Thanks for reporting and digging into the details!

In my opinion, no longer sending a Content-Length header but
instead falling back to chunked transfer encoding is a bug at
least for PHP 7.3, where request #77711 has recently been
back-ported to[1].  For PHP 7.4, this change might be acceptable,
but would at least have to be documented.

However, while having a closer look at this issue, I've noticed
that the current implementation can't really work wrt.
curl_copy_handle()[2], what has to be addressed first, since fixing
that will result in a quite different implementation, which
renders the obvious trivial fix for this issue moot.

> So it seems to be a combination of several CURL versions with
> PHP 7.4

To clarify, this issue affects only libcurl >= 7.56.0, since older
versions still use the "classic" file upload.

[1] <;a=commit;h=17a9f1401aeb35fe1e3657b38102a410d151d42f>
[2] <>
 [2020-01-23 17:10 UTC] sdmarshall73 at gmail dot com
Have you actually tested to see if the files are being received on the remote server? I have an application that uploads files and the files aren't being received on the remote server when using PHP7.4. I wanted to see if this bug is related before filing my own bug report.
 [2020-01-23 18:05 UTC] sdmarshall73 at gmail dot com
My apologies for the previous post ( and this one ). You did mention the files were being removed on the remote side. Seems like the same issue I'm experiencing. I've voted on the bug and I'll just follow this post. Please remove these comments if possible.
 [2020-01-24 13:11 UTC]
-Status: Assigned +Status: Not a bug
 [2020-01-24 13:11 UTC]
For me, using current PHP-7.4 the test script outputs:

Linux, curl 7.64.0:

    POST / HTTP/2
    Accept: */*
    Transfer-Encoding: chunked
    Content-Type: multipart/form-data; boundary=------------------------df0efab1e65ad7e7

Linux, curl 7.68.0:

    POST / HTTP/2
    accept: */*
    content-type: multipart/form-data; boundary=------------------------f55b79edfe19cb82

Windows, curl 7.67.0

    POST / HTTP/2
    accept: */*
    content-type: multipart/form-data; boundary=------------------------1e758282c5b98dfb

So obviously a bug in libcurl which has been fixed in the meantime.
 [2020-02-13 18:53 UTC] andrewscaya at yahoo dot ca
I can confirm that when PHP 7.4 reverts to using chunked file uploads, PHP scripts that were working fine on PHP 7.2 or 7.3, no longer post anything to the server.


<?php var_dump($_FILES); ?>

// Create a cURL handle
$ch = curl_init('');

$cfile = curl_file_create('example.txt','text/plain','test_name');

// Assign POST data
$postdata = ['file' => $cfile];
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);

// Execute the handle
echo curl_exec($ch);


Using cURL 7.68.0 and PHP 7.3

array (size=1)
  'file' =>
    array (size=5)
      'name' => 'test_name' (length=9)
      'type' => 'text/plain' (length=10)
      'tmp_name' => '/tmp/php69gfI5' (length=14)
      'error' => 0
      'size' => 25

POST /index.php HTTP/1.1
Content-Type: multipart/form-data; boundary=------------------------63a96cbe43cf28f8
Content-Length: 212
Accept: */*

Using cURL 7.68.0 and PHP 7.4

array (size=0)

POST /index.php HTTP/1.1
Expect: 100-continue
Content-Type: multipart/form-data; boundary=------------------------46680d0956db253f
Transfer-Encoding: chunked
Accept: */*


Should be the same.
 [2020-02-14 10:55 UTC]
-Status: Not a bug +Status: Re-Opened
PHP Copyright © 2001-2020 The PHP Group
All rights reserved.
Last updated: Thu Feb 27 08:01:27 2020 UTC