php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #60285 setting REPORT_EXIT_STATUS for make test won't work
Submitted: 2011-11-13 18:35 UTC Modified: 2012-11-25 03:51 UTC
Votes:3
Avg. Score:3.7 ± 0.5
Reproduced:3 of 3 (100.0%)
Same Version:1 (33.3%)
Same OS:1 (33.3%)
From: tyrael@php.net Assigned: laruence (profile)
Status: Closed Package: Testing related
PHP Version: trunk-SVN-2011-11-13 (SVN) OS:
Private report: No CVE-ID: None
View Add Comment Developer Edit
Anyone can comment on a bug. Have a simpler test case? Does it work for you on a different platform? Let us know!
Just going to say 'Me too!'? Don't clutter the database with that please !
Your email address:
MUST BE VALID
Solve the problem:
50 + 43 = ?
Subscribe to this entry?

 
 [2011-11-13 18:35 UTC] tyrael@php.net
Description:
------------
when setting up the ci environment, I noticed, that passing the REPORT_EXIT_STATUS 
environment variable to run-tests.php works, but passing the same thing to make 
test won't.
after some debugging with Hannes, we figured out, that the test target in the 
Makefile overuses the error suppression 
operator(http://sunsite.ualberta.ca/Documentation/Gnu/make-
3.79/html_chapter/make_5.html#SEC48).
if you check the test target (http://svn.php.net/viewvc/php/php-
src/trunk/Makefile.global?view=markup#l88) you can see that the whole target is a 
one-liner prefixed with -
which means that anything can fail in the target, it will still report success. :/

the same pattern is used for many of the targets.
Could somebody look into this?

Test script:
---------------
REPORT_EXIT_STATUS=1 make test;
echo $?;

Expected result:
----------------
should print non-zero if there are failed tests.

Actual result:
--------------
always prints 0

Patches

fix-Makefile-return-exit-code (last revision 2012-06-21 03:24 UTC by reeze dot xia at gmail dot com)

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2011-11-13 18:45 UTC] tyrael@php.net
I just checked, it was introduced way back in 2003:
http://svn.php.net/viewvc?view=revision&revision=135844
 [2012-06-21 03:21 UTC] reeze dot xia at gmail dot com
Yes, I got the same problem,  I use Travis to test my extension, but make test 
didn't report failed test correctly. 

http://travis-ci.org/#!/reeze/php-sundown/jobs/1670528

That commit ignore the status code, the lastest version add another command 
after run-test: 

rm $(top_builddir)/tmp-php.ini; \

This makes it never return exit code.
--------------------------
 [2012-11-25 03:51 UTC] laruence@php.net
-Status: Open +Status: Closed -Assigned To: +Assigned To: laruence
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved.
Last updated: Fri Apr 19 18:01:28 2024 UTC