php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login
Bug #60285 setting REPORT_EXIT_STATUS for make test won't work
Submitted: 2011-11-13 18:35 UTC Modified: 2012-11-25 03:51 UTC
Votes:3
Avg. Score:3.7 ± 0.5
Reproduced:3 of 3 (100.0%)
Same Version:1 (33.3%)
Same OS:1 (33.3%)
From: tyrael@php.net Assigned: laruence (profile)
Status: Closed Package: Testing related
PHP Version: trunk-SVN-2011-11-13 (SVN) OS:
Private report: No CVE-ID: None
View Add Comment Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
You can add a comment by following this link or if you reported this bug, you can edit this bug over here.
(description)
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: tyrael@php.net
New email:
PHP Version: OS:

 

 [2011-11-13 18:35 UTC] tyrael@php.net
Description:
------------
when setting up the ci environment, I noticed, that passing the REPORT_EXIT_STATUS 
environment variable to run-tests.php works, but passing the same thing to make 
test won't.
after some debugging with Hannes, we figured out, that the test target in the 
Makefile overuses the error suppression 
operator(http://sunsite.ualberta.ca/Documentation/Gnu/make-
3.79/html_chapter/make_5.html#SEC48).
if you check the test target (http://svn.php.net/viewvc/php/php-
src/trunk/Makefile.global?view=markup#l88) you can see that the whole target is a 
one-liner prefixed with -
which means that anything can fail in the target, it will still report success. :/

the same pattern is used for many of the targets.
Could somebody look into this?

Test script:
---------------
REPORT_EXIT_STATUS=1 make test;
echo $?;

Expected result:
----------------
should print non-zero if there are failed tests.

Actual result:
--------------
always prints 0

Patches

fix-Makefile-return-exit-code (last revision 2012-06-21 03:24 UTC by reeze dot xia at gmail dot com)

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2011-11-13 18:45 UTC] tyrael@php.net
I just checked, it was introduced way back in 2003:
http://svn.php.net/viewvc?view=revision&revision=135844
 [2012-06-21 03:21 UTC] reeze dot xia at gmail dot com
Yes, I got the same problem,  I use Travis to test my extension, but make test 
didn't report failed test correctly. 

http://travis-ci.org/#!/reeze/php-sundown/jobs/1670528

That commit ignore the status code, the lastest version add another command 
after run-test: 

rm $(top_builddir)/tmp-php.ini; \

This makes it never return exit code.
--------------------------
 [2012-11-25 03:51 UTC] laruence@php.net
-Status: Open +Status: Closed -Assigned To: +Assigned To: laruence
 
PHP Copyright © 2001-2019 The PHP Group
All rights reserved.
Last updated: Mon Jun 17 09:01:36 2019 UTC