|  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #60285 setting REPORT_EXIT_STATUS for make test won't work
Submitted: 2011-11-13 18:35 UTC Modified: 2012-11-25 03:51 UTC
Avg. Score:3.7 ± 0.5
Reproduced:3 of 3 (100.0%)
Same Version:1 (33.3%)
Same OS:1 (33.3%)
From: Assigned: laruence (profile)
Status: Closed Package: Testing related
PHP Version: trunk-SVN-2011-11-13 (SVN) OS:
Private report: No CVE-ID: None
 [2011-11-13 18:35 UTC]
when setting up the ci environment, I noticed, that passing the REPORT_EXIT_STATUS 
environment variable to run-tests.php works, but passing the same thing to make 
test won't.
after some debugging with Hannes, we figured out, that the test target in the 
Makefile overuses the error suppression 
if you check the test target (
src/trunk/ you can see that the whole target is a 
one-liner prefixed with -
which means that anything can fail in the target, it will still report success. :/

the same pattern is used for many of the targets.
Could somebody look into this?

Test script:
echo $?;

Expected result:
should print non-zero if there are failed tests.

Actual result:
always prints 0


fix-Makefile-return-exit-code (last revision 2012-06-21 03:24 UTC by reeze dot xia at gmail dot com)

Add a Patch

Pull Requests

Add a Pull Request


AllCommentsChangesGit/SVN commitsRelated reports
 [2011-11-13 18:45 UTC]
I just checked, it was introduced way back in 2003:
 [2012-06-21 03:21 UTC] reeze dot xia at gmail dot com
Yes, I got the same problem,  I use Travis to test my extension, but make test 
didn't report failed test correctly.!/reeze/php-sundown/jobs/1670528

That commit ignore the status code, the lastest version add another command 
after run-test: 

rm $(top_builddir)/tmp-php.ini; \

This makes it never return exit code.
 [2012-11-25 03:51 UTC]
-Status: Open +Status: Closed -Assigned To: +Assigned To: laruence
PHP Copyright © 2001-2020 The PHP Group
All rights reserved.
Last updated: Sun Oct 25 22:01:24 2020 UTC