|  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #60285 setting REPORT_EXIT_STATUS for make test won't work
Submitted: 2011-11-13 18:35 UTC Modified: 2012-11-25 03:51 UTC
Avg. Score:3.7 ± 0.5
Reproduced:3 of 3 (100.0%)
Same Version:1 (33.3%)
Same OS:1 (33.3%)
From: Assigned: laruence (profile)
Status: Closed Package: Testing related
PHP Version: trunk-SVN-2011-11-13 (SVN) OS:
Private report: No CVE-ID: None
View Add Comment Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
You can add a comment by following this link or if you reported this bug, you can edit this bug over here.
Block user comment
Status: Assign to:
Bug Type:
New email:
PHP Version: OS:


 [2011-11-13 18:35 UTC]
when setting up the ci environment, I noticed, that passing the REPORT_EXIT_STATUS 
environment variable to run-tests.php works, but passing the same thing to make 
test won't.
after some debugging with Hannes, we figured out, that the test target in the 
Makefile overuses the error suppression 
if you check the test target (
src/trunk/ you can see that the whole target is a 
one-liner prefixed with -
which means that anything can fail in the target, it will still report success. :/

the same pattern is used for many of the targets.
Could somebody look into this?

Test script:
echo $?;

Expected result:
should print non-zero if there are failed tests.

Actual result:
always prints 0


fix-Makefile-return-exit-code (last revision 2012-06-21 03:24 UTC by reeze dot xia at gmail dot com)

Add a Patch

Pull Requests

Add a Pull Request


AllCommentsChangesGit/SVN commitsRelated reports
 [2011-11-13 18:45 UTC]
I just checked, it was introduced way back in 2003:
 [2012-06-21 03:21 UTC] reeze dot xia at gmail dot com
Yes, I got the same problem,  I use Travis to test my extension, but make test 
didn't report failed test correctly.!/reeze/php-sundown/jobs/1670528

That commit ignore the status code, the lastest version add another command 
after run-test: 

rm $(top_builddir)/tmp-php.ini; \

This makes it never return exit code.
 [2012-11-25 03:51 UTC]
-Status: Open +Status: Closed -Assigned To: +Assigned To: laruence
PHP Copyright © 2001-2021 The PHP Group
All rights reserved.
Last updated: Tue Jun 22 02:01:23 2021 UTC