|  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #60285 setting REPORT_EXIT_STATUS for make test won't work
Submitted: 2011-11-13 18:35 UTC Modified: 2012-11-25 03:51 UTC
Avg. Score:3.7 ± 0.5
Reproduced:3 of 3 (100.0%)
Same Version:1 (33.3%)
Same OS:1 (33.3%)
From: Assigned: laruence (profile)
Status: Closed Package: Testing related
PHP Version: trunk-SVN-2011-11-13 (SVN) OS:
Private report: No CVE-ID: None
View Add Comment Developer Edit
Anyone can comment on a bug. Have a simpler test case? Does it work for you on a different platform? Let us know!
Just going to say 'Me too!'? Don't clutter the database with that please !
Your email address:
Solve the problem:
26 - 9 = ?
Subscribe to this entry?

 [2011-11-13 18:35 UTC]
when setting up the ci environment, I noticed, that passing the REPORT_EXIT_STATUS 
environment variable to run-tests.php works, but passing the same thing to make 
test won't.
after some debugging with Hannes, we figured out, that the test target in the 
Makefile overuses the error suppression 
if you check the test target (
src/trunk/ you can see that the whole target is a 
one-liner prefixed with -
which means that anything can fail in the target, it will still report success. :/

the same pattern is used for many of the targets.
Could somebody look into this?

Test script:
echo $?;

Expected result:
should print non-zero if there are failed tests.

Actual result:
always prints 0


fix-Makefile-return-exit-code (last revision 2012-06-21 03:24 UTC by reeze dot xia at gmail dot com)

Add a Patch

Pull Requests

Add a Pull Request


AllCommentsChangesGit/SVN commitsRelated reports
 [2011-11-13 18:45 UTC]
I just checked, it was introduced way back in 2003:
 [2012-06-21 03:21 UTC] reeze dot xia at gmail dot com
Yes, I got the same problem,  I use Travis to test my extension, but make test 
didn't report failed test correctly.!/reeze/php-sundown/jobs/1670528

That commit ignore the status code, the lastest version add another command 
after run-test: 

rm $(top_builddir)/tmp-php.ini; \

This makes it never return exit code.
 [2012-11-25 03:51 UTC]
-Status: Open +Status: Closed -Assigned To: +Assigned To: laruence
PHP Copyright © 2001-2021 The PHP Group
All rights reserved.
Last updated: Wed Jun 23 05:01:23 2021 UTC