php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #73485 mail() adds a lone CR or LF
Submitted: 2016-11-10 02:22 UTC Modified: 2017-01-08 04:22 UTC
Votes:1
Avg. Score:5.0 ± 0.0
Reproduced:1 of 1 (100.0%)
Same Version:1 (100.0%)
Same OS:1 (100.0%)
From: sherwoodsports at gmail dot com Assigned: yohgaki (profile)
Status: No Feedback Package: Mail related
PHP Version: 7.0.X OS: All
Private report: No CVE-ID: None
Have you experienced this issue?
Rate the importance of this bug to you:

 [2016-11-10 02:22 UTC] sherwoodsports at gmail dot com
Description:
------------
Mail sent with the mail() function in PHP 7.0.9 or 7.0.11 results in a 550 error from mail servers that does not manifest in mails sent with the mail() function in PHP 5.3.24.

The situation results in the receiving mail server to throw the following error during the DATA transmission of the mail message:

550+5.6.0+Lone+CR+or+LF+in+headers+(see+RFC2822+section+2.2)

SMTP servers vary as it is a configuration issue to block incoming mail that exhibits this issue.  The fix is likely to compare output of mail() for 5.3.24 vs. mail for 7.0.9 or 7.0.11 and resolve the "extra" CR or LF character addition.

Expected result:
----------------
Sending from 5.3.24 (to server which identifies issue):

2016-11-10 00:11:35 173.203.187.1 OutboundConnectionResponse SMTPSVC1 xxxx - 25 - - 220+mx1.emailsrvr.com+ESMTP+-+VA+Code+Section+18.2-152.3:1+forbids+sending+spam+through+this+system 0 0 99 0 16 SMTP - - - -
2016-11-10 00:11:35 173.203.187.1 OutboundConnectionCommand SMTPSVC1 xxxx - 25 EHLO - xxxx 0 0 4 0 16 SMTP - - - -
2016-11-10 00:11:35 173.203.187.1 OutboundConnectionResponse SMTPSVC1 xxxx - 25 - - 250-smtp5.gate.iad3a.rsapps.net+says+EHLO+to+xxx.xxx.xxx.xxx:60802 0 0 65 0 47 SMTP - - - -
2016-11-10 00:11:35 173.203.187.1 OutboundConnectionCommand SMTPSVC1 xxxx - 25 MAIL - FROM:<name@domain.com>+SIZE=740 0 0 4 0 47 SMTP - - - -
2016-11-10 00:11:35 173.203.187.1 OutboundConnectionResponse SMTPSVC1 xxxx - 25 - - 250+2.1.0+Ok 0 0 12 0 172 SMTP - - - -
2016-11-10 00:11:35 173.203.187.1 OutboundConnectionCommand SMTPSVC1 xxxx - 25 RCPT - TO:<sendtoname@sendtodomain.com> 0 0 4 0 172 SMTP - - - -
2016-11-10 00:11:35 173.203.187.1 OutboundConnectionResponse SMTPSVC1 xxxx - 25 - - 250+2.1.5+Ok 0 0 12 0 219 SMTP - - - -
2016-11-10 00:11:35 173.203.187.1 OutboundConnectionCommand SMTPSVC1 xxxx - 25 DATA - - 0 0 4 0 219 SMTP - - - -
2016-11-10 00:11:35 173.203.187.1 OutboundConnectionResponse SMTPSVC1 xxxx - 25 - - 354+3.0.0+continue.++finished+with+"\r\n.\r\n" 0 0 46 0 250 SMTP - - - -
2016-11-10 00:11:35 173.203.187.1 OutboundConnectionResponse SMTPSVC1 xxxx - 25 - - 250+2.0.0+Ok:+queued+as+D4/38-19322-24BB3285 0 0 44 0 468 SMTP - - - -
2016-11-10 00:11:35 173.203.187.1 OutboundConnectionCommand SMTPSVC1 xxxx - 25 QUIT - - 0 0 4 0 468 SMTP - - - -
2016-11-10 00:11:35 173.203.187.1 OutboundConnectionResponse SMTPSVC1 xxxx - 25 - - 221+2.3.0+smtp5.gate.iad3a.rsapps.net+closing+connection 0 0 56 0 484 SMTP - - - -


Actual result:
--------------
Sending from 7.0.11 (to server which identifies issue):

2016-11-10 00:00:00 173.203.187.1 OutboundConnectionResponse SMTPSVC1 xxxx - 25 - - 220+mx1.emailsrvr.com+ESMTP+-+VA+Code+Section+18.2-152.3:1+forbids+sending+spam+through+this+system 0 0 99 0 16 SMTP - - - -
2016-11-10 00:00:00 173.203.187.1 OutboundConnectionCommand SMTPSVC1 xxxx - 25 EHLO - xxxx 0 0 4 0 16 SMTP - - - -
2016-11-10 00:00:00 173.203.187.1 OutboundConnectionResponse SMTPSVC1 xxxx - 25 - - 250-smtp52.gate.iad3a.rsapps.net+says+EHLO+to+xxx.xxx.xxx.xxx:60785 0 0 66 0 47 SMTP - - - -
2016-11-10 00:00:00 173.203.187.1 OutboundConnectionCommand SMTPSVC1 xxxx - 25 MAIL - FROM:<name@domain.com>+SIZE=743 0 0 4 0 47 SMTP - - - -
2016-11-10 00:00:00 173.203.187.1 OutboundConnectionResponse SMTPSVC1 xxxx - 25 - - 250+2.1.0+Ok 0 0 12 0 78 SMTP - - - -
2016-11-10 00:00:00 173.203.187.1 OutboundConnectionCommand SMTPSVC1 xxxx - 25 RCPT - TO:<sendtoname@sendtodomain.com> 0 0 4 0 78 SMTP - - - -
2016-11-10 00:00:00 173.203.187.1 OutboundConnectionResponse SMTPSVC1 xxxx - 25 - - 250+2.1.5+Ok 0 0 12 0 140 SMTP - - - -
2016-11-10 00:00:00 173.203.187.1 OutboundConnectionCommand SMTPSVC1 xxxx - 25 DATA - - 0 0 4 0 140 SMTP - - - -
2016-11-10 00:00:00 173.203.187.1 OutboundConnectionResponse SMTPSVC1 xxxx - 25 - - 354+3.0.0+continue.++finished+with+"\r\n.\r\n" 0 0 46 0 156 SMTP - - - -
2016-11-10 00:00:00 173.203.187.1 OutboundConnectionResponse SMTPSVC1 xxxx - 25 - - 550+5.6.0+Lone+CR+or+LF+in+headers+(see+RFC2822+section+2.2) 0 0 60 0 172 SMTP - - - -
2016-11-10 00:00:00 173.203.187.1 OutboundConnectionCommand SMTPSVC1 xxxx - 25 QUIT - - 0 0 4 0 187 SMTP - - - -
2016-11-10 00:00:00 173.203.187.1 OutboundConnectionResponse SMTPSVC1 xxxx - 25 - - 221+2.3.0+smtp52.gate.iad3a.rsapps.net+closing+connection 0 0 57 0 203 SMTP - - - -



Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2016-11-10 15:02 UTC] sherwoodsports at gmail dot com
-Operating System: +Operating System: All -PHP Version: 7.0Git-2016-11-10 (snap) +PHP Version: 7.0.X
 [2016-11-10 15:02 UTC] sherwoodsports at gmail dot com
Further research shows others having no problem with PHP 5.6.
Changing affected PHP version to 7.0.X.
 [2016-11-11 17:40 UTC] mattcoz at yahoo dot com
I think this is the same as my issue. There seems to be an extra character before the additional headers. For example:

mail("to@email.com", "Subject", "Message", "From: from@email.com");

To: <to@email.com>
X-PHP-Originating-Script: 0:test.php
 From: from@email.com
From: <from@email.com>

An additional From header is added and this causes it to be blocked when sent to gmail addresses. On the other hand, if I put a dummy header before it:

mail("to@email.com", "Subject", "Message", "X-PHP-Fix: x\r\nFrom: from@email.com");

To: <to@email.com>
X-PHP-Originating-Script: 0:test.php
 X-PHP-Fix: x
From: <from@email.com>

This is sent successfully to gmail addresses. My guess is that this is the lone CR or LF that is causing his emails to fail. I have one server running 7.0.10 that is affected and one running 5.3.13 that is not affected, which matches his experience.
 [2016-11-12 07:02 UTC] rasmus@php.net
-Status: Open +Status: Analyzed
 [2016-11-12 07:02 UTC] rasmus@php.net
I don't think this is PHP doing this. At least not directly. Which MTA are you using? The first thing to understand here is that by default PHP just invokes an external MTA to send an email with the mail() function on non-Windows machines. The program that is run is defined by your sendmail_path ini setting. This also means we can override it to examine what exactly PHP is sending to the MTA program.

Here is a test. Create this trivial PHP program:

#!/usr/local/bin/php
<?php
file_put_contents("/tmp/test.log", file_get_contents("php://stdin"), FILE_APPEND);

I called mine /home/rasmus/sm and I made it executable (chmod +x ./sm)

Then I tried your example script of:

<?php
mail("to@email.com", "Subject", "Message", "From: from@email.com");

I ran it like this:

php -d sendmail_path="/home/rasmus/sm" m

The output I see in /tmp/test.log is:

To: to@email.com
Subject: Subject
From: from@email.com

Message

What I think is happening here, at least for mattcoz at yahoo dot com is that the user invoking the MTA is not conifgured to be a trusted user and as such is not allowed to override the From address. The MTA inserts the space in front of it and adds its own From line.

For sherwoodsports at gmail dot com, what is your mail() line that is producing that error? As you can see from my minimal test it doesn't appear that PHP is inserting an extra cr-lf in the headers. Again, it is likely some sort of interaction with your MTA.
 [2016-11-12 17:21 UTC] mattcoz at yahoo dot com
I'm running Windows, so the MTA is not an issue. Also, I have no problem overriding the from address. I think I found where the problem might really be though. My server had the mail.add_x_header preference set to On for some reason. When I switch it to Off I no longer get the space before the rest of my headers. I'll open a new bug report for this specific issue and leave it Off on my server because I really don't need it.
 [2016-11-24 03:03 UTC] yohgaki@php.net
-Status: Analyzed +Status: Not a bug
 [2016-11-24 03:03 UTC] yohgaki@php.net
We have test cases for multiple headers by additional_headers parameter.
http://lxr.php.net/xref/PHP-7.0/ext/standard/tests/mail/mail_basic6.phpt

> mail("to@email.com", "Subject", "Message", "X-PHP-Fix: x\r\nFrom: from@email.com");

This shouldn't produce following headers.

> To: <to@email.com>
> X-PHP-Originating-Script: 0:test.php
>  From: from@email.com
> From: <from@email.com>

I suppose this is your MTA issue.

If you think this is a bug in PHP, try to add bug case to 
http://lxr.php.net/xref/PHP-7.0/ext/standard/tests/mail/mail_basic6.phpt

You can run the test in PHP source build like

TESTS=ext/standard/tests/mail/mail_basic6.phpt make test

Attach modified mail_basic6.phpt file and reopen this bug.
 [2016-11-24 05:46 UTC] sherwoodsports at gmail dot com
Hold up...  the comments posted are not related to the original report.  Please do not identify this bug with those comments then mark this bug as "not a bug" due to your analysis of the issue identified in the comments.
 [2016-11-24 05:55 UTC] mattcoz at yahoo dot com
I opened a new bug for my issue: https://bugs.php.net/bug.php?id=73507
 [2016-11-24 07:51 UTC] yohgaki@php.net
Like I mentioned, you should provide "bug case", not creating new bug report.

PHP is not the only program modify mail header, so you should provide case that PHP produces malformed mail headers.

BTW

To: <to@email.com>
X-PHP-Originating-Script: 0:test.php
 From: from@email.com
From: <from@email.com>  <--- PHP never adds this 

by 

mail("to@email.com", "Subject", "Message", "From: from@email.com");


Current mail() accepts multiline header, i.e. it allows too long line folding by  "\r\n ", "\r\n\t", but it never split lines automatically.
 [2016-11-24 07:57 UTC] yohgaki@php.net
It trims unneeded chars at the first and end, detects malformed newlines, but it never add modified headers by parsing addtional_headers.

If you have problem, please try to reproduce by modifying the phpt file that I've mentioned.
 [2016-11-24 08:50 UTC] yohgaki@php.net
OK. Back to original report.

Could you modify the test script and attach the modified test script (the phpt) so that the problem can be seen as plain text output?

OS should not be "All". Are you using Windows or UNIX like OS? (Windows and UNIX uses different code to send mails) Please specify what OS you are using.
 [2016-11-24 15:56 UTC] sherwoodsports at gmail dot com
Again, your comment is unrelated to the original bug report.  Please re-open this bug.  I am unable to compare output between 5.3 and 7.0.  This is what needs to be done to determine if this original report is a bug or not.
 [2016-11-24 16:12 UTC] Sherwoodsports at gmail dot com
I will modify and will send the output as soon as I can but am unable to for a day or two.
 [2016-11-24 16:13 UTC] Sherwoodsports at gmail dot com
This is on Windows server 2008.
 [2016-11-25 07:05 UTC] yohgaki@php.net
-Status: Not a bug +Status: Feedback -Assigned To: +Assigned To: yohgaki
 [2016-11-25 07:05 UTC] yohgaki@php.net
I reopened this bug for the time being.
It seems this problem is due to MTA/UMA newline conversion which is not a bug of PHP.

Recent MTA/UMA do _not_ convert LF to CR/LF and this could results in errors like you have. It seems you are experiencing this issue and if this is the case, you should fix your code. i.e. Use CR/LF for new lines as defined in RFC.
 [2016-11-25 15:21 UTC] sherwoodsports at gmail dot com
-Status: Feedback +Status: Assigned
 [2016-11-25 15:21 UTC] sherwoodsports at gmail dot com
Thank you for re-opening.  

A manipulated MTA/UMA likely would not explain why the older version of PHP works, but the newer version does not.  

Also, all CRLF's used in the code are using PHP_EOL and are not using manual ('\n\r') characters to end lines.  No single \r or \n characters are used, therefore are not subject to the non-conversion issue you suggest.  This also supports the fact that the older PHP versions prior to 7 works without issues, but 7 does have issues.
 [2016-12-12 07:13 UTC] yohgaki@php.net
-Status: Assigned +Status: Feedback
 [2016-12-12 07:13 UTC] yohgaki@php.net
Are you sure that you are using the same MTA (mail server) and UMA (sendmail binary) for old and new PHPs? If not, MTA/UMA behavior change is the cause most likely.

If you are using the same MTA/UMA, please modify following test script and reproduce issue.

https://github.com/php/php-src/blob/master/ext/standard/tests/mail/mail_basic.phpt
 [2016-12-12 14:27 UTC] sherwoodsports at gmail dot com
Unfortunately, I'm in a Windows Server 2008 R2 environment.  After reading it, that script indicates right away that it won't run on Windows.

I use the IIS web server to swap PHP versions that IIS is using for any given web site.  The SMTP server is Microsoft CDO (basically Microsoft Exchange Libraries) and it has never changed (this library is no longer updated via Microsoft) between various server web languages (ASP, PHP 5.3.24) and has only experienced issues with PHP 7 and above.  All that said, it would be difficult to say a library that has never changed is suddenly sensitive to PHP 7 (I still have PHP 5 installed and it still doesn't fail).

Do you have a script that will run on Windows?  Is there a command that fails on Windows that I can modify in that script such that it will run successfully?
 [2016-12-25 04:22 UTC] php-bugs at lists dot php dot net
No feedback was provided. The bug is being suspended because
we assume that you are no longer experiencing the problem.
If this is not the case and you are able to provide the
information that was requested earlier, please do so and
change the status of the bug back to "Re-Opened". Thank you.
 [2016-12-25 14:25 UTC] requinix@php.net
-Status: No Feedback +Status: Feedback
 [2016-12-25 14:25 UTC] requinix@php.net
You can configure SMTP to dump to a pickup directory:
https://technet.microsoft.com/en-us/library/cc772058(v=ws.10).aspx

Alter the path to $outFile in the test script appropriately, comment out the SKIPIF code, and you should be able to run the test code.
 [2017-01-08 04:22 UTC] php-bugs at lists dot php dot net
No feedback was provided. The bug is being suspended because
we assume that you are no longer experiencing the problem.
If this is not the case and you are able to provide the
information that was requested earlier, please do so and
change the status of the bug back to "Re-Opened". Thank you.
 
PHP Copyright © 2001-2020 The PHP Group
All rights reserved.
Last updated: Fri Nov 27 18:01:25 2020 UTC