This bug was raised in 2015 but closed. 

In those days there were mirrors which worked so it was much less of a problem for people with automated builds.

That no longer applies so I believe it is an issue for which re-assessment would be beneficial.

Just tried my local system and it worked fine, so there's something else up.

$ wget --verbose https://www.php.net/distributions/php-8.0.6.tar.gz
--2021-05-19 19:47:43--  https://www.php.net/distributions/php-8.0.6.tar.gz
Resolving www.php.net (www.php.net)... 185.85.0.29, 2a02:cb40:200::1ad
Connecting to www.php.net (www.php.net)|185.85.0.29|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 17184903 (16M) [application/octet-stream]
Saving to: ‘php-8.0.6.tar.gz’

php-8.0.6.tar.gz                        100%[===============================================================================>]  16.39M  5.53MB/s    in 3.0s    

2021-05-19 19:47:51 (5.53 MB/s) - ‘php-8.0.6.tar.gz’ saved [17184903/17184903]


Sadly, `wget --verbose` doesn't seem to give a lot of detail.  I wonder if using curl works.  If not, maybe looking at the output of `curl --verbose` would provide more insight.

curl --verbose -O https://www.php.net/distributions/php-8.0.6.tar.gz

Oh, and to your first item: Yes, the mirrors have all been retired. There IS a CDN in front of the main php.net server, so the effect is much the same, and it's possible the 503 errors you're getting are from them (which will make diagnosis more tricky).

As a short term workaround, you can try pulling from github: https://github.com/php/web-php-distributions/ ) which will present a different CDN stack and may work around your issue (which, noticing your .za address, feels more likely to be CDN related).

Cloning the entire repo would be a bit much, but you can grab individual.  E.g. for 8.0.6, you should be able to wget from: https://github.com/php/web-php-distributions/raw/master/php-8.0.6.tar.gz

In the longer term, I would like to figure out what's going on so that the more generalized path to downloading images can be fixed.  If you wouldn't mind sharing that curl verbose output, it might point to something.

curl works a treat - thank you!

For info, curl verbose output follows....

curl --verbose -O https://www.php.net/distributions/php-8.0.6.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* About to connect() to www.php.net port 443 (#0)
*   Trying 185.85.0.29...
* Connected to www.php.net (185.85.0.29) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* 	subject: CN=*.php.net
* 	start date: May 18 10:04:38 2021 GMT
* 	expire date: May 18 10:04:38 2022 GMT
* 	common name: *.php.net
* 	issuer: CN=Certum Domain Validation CA SHA2,OU=Certum Certification Authority,O=Unizeto Technologies S.A.,C=PL
> GET /distributions/php-8.0.6.tar.gz HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.php.net
> Accept: */*
> 
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0< HTTP/1.1 200 OK
< Server: myracloud
< Date: Thu, 20 May 2021 17:56:31 GMT
< Content-Type: application/octet-stream
< Content-Length: 17184903
< Connection: keep-alive
< Last-Modified: Tue, 04 May 2021 18:00:08 GMT
< ETag: "60918ba8-1063887"
< X-CDN: 1
< Accept-Ranges: bytes
< 
{ [data not shown]
100 16.3M  100 16.3M    0     0   698k      0  0:00:24  0:00:24 --:--:--  796k
* Connection #0 to host www.php.net left intact

Just for info... 

wget --verbose

provided no more information than I originally provided (and still fails in the same way)

Great news that curl works. Hopefully it's not too much trouble to use it rather than wget.

As to wget... I'm not sure what to think here.  Your curl --verbose output largely matches mine (apart from minor differences which could be down to using slightly different versions), so I'm not sure the CDN is to blame. Something that wget does different just... borks.

What version of wget are you using, and on what distro? Perhaps I can repro the bug in a container.

Is this still an issue, or was it maybe related to the certificate
expiration problem, which should have been resolved now.

Nah. This should have nothing to do with the cert issues on pear/pecl.

Indeed, as you can see in the pasted curl --verbose output, the ticket submitter was seeing an expiration date a year away:

expire date: May 18 10:04:38 2022 GMT

I am getting the same problem on CentOS 7.9 64bit OS

date
Wed May 26 04:57:08 UTC 2021

wget https://www.php.net/distributions/php-8.0.6.tar.gz -O /dev/null
--2021-05-26 04:57:15--  https://www.php.net/distributions/php-8.0.6.tar.gz
Resolving www.php.net (www.php.net)... 2a02:cb40:200::1ad, 185.85.0.29
Connecting to www.php.net (www.php.net)|2a02:cb40:200::1ad|:443... connected.
HTTP request sent, awaiting response... 503 Service Temporarily Unavailable
2021-05-26 04:57:16 ERROR 503: Service Temporarily Unavailable.

wget -4 https://www.php.net/distributions/php-8.0.6.tar.gz -O /dev/null
--2021-05-26 04:57:59--  https://www.php.net/distributions/php-8.0.6.tar.gz
Resolving www.php.net (www.php.net)... 185.85.0.29
Connecting to www.php.net (www.php.net)|185.85.0.29|:443... connected.
HTTP request sent, awaiting response... 503 Service Temporarily Unavailable
2021-05-26 04:57:59 ERROR 503: Service Temporarily Unavailable.

curl -4Iv https://www.php.net/distributions/php-8.0.6.tar.gz
* About to connect() to www.php.net port 443 (#0)
*   Trying 185.85.0.29...
* Connected to www.php.net (185.85.0.29) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=*.php.net
*       start date: May 18 10:04:38 2021 GMT
*       expire date: May 18 10:04:38 2022 GMT
*       common name: *.php.net
*       issuer: CN=Certum Domain Validation CA SHA2,OU=Certum Certification Authority,O=Unizeto Technologies S.A.,C=PL
> HEAD /distributions/php-8.0.6.tar.gz HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.php.net
> Accept: */*
> 
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Server: myracloud
Server: myracloud
< Date: Wed, 26 May 2021 04:58:26 GMT
Date: Wed, 26 May 2021 04:58:26 GMT
< Content-Type: application/octet-stream
Content-Type: application/octet-stream
< Content-Length: 17184903
Content-Length: 17184903
< Connection: keep-alive
Connection: keep-alive
< Last-Modified: Tue, 04 May 2021 18:00:08 GMT
Last-Modified: Tue, 04 May 2021 18:00:08 GMT
< ETag: "60918ba8-1063887"
ETag: "60918ba8-1063887"
< X-CDN: 1
X-CDN: 1
< Accept-Ranges: bytes
Accept-Ranges: bytes

< 
* Connection #0 to host www.php.net left intact

wget version

wget -V
GNU Wget 1.14 built on linux-gnu.

+digest +https +ipv6 +iri +large-file +nls +ntlm +opie +ssl/openssl 

Wgetrc: 
    /etc/wgetrc (system)
Locale: /usr/share/locale 
Compile: gcc -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/etc/wgetrc" 
    -DLOCALEDIR="/usr/share/locale" -I. -I../lib -I../lib -O2 -g -pipe 
    -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong 
    --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic 
Link: gcc -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions 
    -fstack-protector-strong --param=ssp-buffer-size=4 
    -grecord-gcc-switches -m64 -mtune=generic -lssl -lcrypto 
    /usr/lib64/libssl.so /usr/lib64/libcrypto.so /usr/lib64/libz.so 
    -ldl -lz -lz -lidn -luuid -lpcre ftp-opie.o openssl.o http-ntlm.o 
    ../lib/libgnu.a 

Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Originally written by Hrvoje Niksic <hniksic@xemacs.org>.
Please send bug reports and questions to <bug-wget@gnu.org>.

on Upcloud VPS provider KVM server

curl -s https://ipinfo.io
{
  "ip": "152.44.46.xxx",
  "city": "San Francisco",
  "region": "California",
  "country": "US",
  "loc": "37.7749,-122.4194",
  "org": "AS25697 UpCloud USA Inc",
  "postal": "94103",
  "timezone": "America/Los_Angeles",
  "readme": "https://ipinfo.io/missingauth"
}

Seems it might be wget version related ? on CentOS 7.9 64bit wget 1.14 is default. But once I upgrade to wget 1.20.3 it works!

wget -4 https://www.php.net/distributions/php-8.0.6.tar.gz -O /dev/null
--2021-05-26 05:25:05--  https://www.php.net/distributions/php-8.0.6.tar.gz
Resolving www.php.net... 185.85.0.29
Connecting to www.php.net|185.85.0.29|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 17184903 (16M) [application/octet-stream]
Saving to: ‘/dev/null’

/dev/null                                           100%[=================================================================================================================>]  16.39M  9.61MB/s    in 1.7s    

2021-05-26 05:25:08 (9.61 MB/s) - ‘/dev/null’ saved [17184903/17184903]


wget -V
GNU Wget 1.20.3 built on linux-gnu.

-cares +digest -gpgme +https +ipv6 -iri +large-file -metalink +nls 
+ntlm +opie -psl +ssl/openssl 

Wgetrc: 
    /usr/local/etc/wgetrc (system)
Locale: 
    /usr/local/share/locale 
Compile: 
    ccache gcc -std=gnu11 -DHAVE_CONFIG_H 
    -DSYSTEM_WGETRC="/usr/local/etc/wgetrc" 
    -DLOCALEDIR="/usr/local/share/locale" -I. -I../lib -I../lib -I 
    /usr/local/include -DHAVE_LIBSSL -DNDEBUG -O2 -g -pipe -Wall 
    -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong 
    --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic 
Link: 
    ccache gcc -std=gnu11 -I /usr/local/include -DHAVE_LIBSSL -DNDEBUG 
    -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions 
    -fstack-protector-strong --param=ssp-buffer-size=4 
    -grecord-gcc-switches -m64 -mtune=generic -L /usr/local/lib -lpcre 
    -luuid -lssl -lcrypto -lz ftp-opie.o openssl.o http-ntlm.o 
    ../lib/libgnu.a 

Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Originally written by Hrvoje Niksic <hniksic@xemacs.org>.
Please send bug reports and questions to <bug-wget@gnu.org>.

Well, reopening, although this doesn't look like a php.net issue.

unless php.net CDN firewall is blocking requests with user agent containing Wget/1.1 which would allow Wget/1.2 to download?

more info wget 1.14 is hitting php.net security check page see debug wget output at https://gist.github.com/centminmod/b97a953a04423867ec89d4b6ee801b5b#not-working-wget-114-default-centos-7

while wget 1.20.3 is working and not hitting security check page from wget default output https://gist.github.com/centminmod/b97a953a04423867ec89d4b6ee801b5b#working-wget-1203

wget 1.14 503 is hitting a HTML page with title <title>Security Check</title> and what triggered it is php.net 

<p class="error-inSkipping 512 bytes of body: [fo"><span class="what">Host</span> <span class="status status-working">Working</span></p></div></div></div><div class="row clearfix"><div class="error-desc"><div class="col one-half"><div class="error-desc-text what-happened"><h2>What happened?</h2><p>You ran into a security check to verify the validity of your request.</p></div></div><div class="col one-half"><div class="error-desc-text what-do"><h2>What can I do?</h2><h3>If you are a visitor of this website:</h3><p>You must confirm that you are human.</p>Skipping 128 bytes of body: [<h3>If you are the owner of this website:</h3><p>Please check your security settings.</p>

Hello,

it seems like some requests have been blocked by the CDN due to an attack happening earlier consisting of similar requests.

This issue should now be resolved, your build should run normally now. Sorry for the inconvenience.

Thanks for the update Moritz!

Just to add data, my (working) wget version was 1.20.3 so that jibes with eva2000bugs' observations, and I guess the trigger was (at least partially) UA based.

I'm going to close this now since the cause has been identified and the issue has been resolved.

Fred, please reopen if you still have trouble with wget.

Thank you so much for your help and my apologies for going off the radar, I have only just seen this. 

I have switched to curl and have had no further problems.

We noticed this 503 in trying to update the PHP CloudFoundry buildpack (https://github.com/cloudfoundry/php-buildpack) via automation.

After doing some debugging, we determined that the 503 was due to being flagged as a potentially-malicious bot by the PHP CDN. (Perhaps they upped security against DDOS due to the recent log4shell exploit? Though I'd expect their server uses, well, PHP and not Java).

In either case, using the GitHub-based URL proposed earlier in this thread should work for now (e.g. https://github.com/php/web-php-distributions/raw/master/php-7.4.27.tar.gz), though it would be nice to hear from PHP maintainers on why this started happening. It used to work in Dec 2021, looking at our build history.

@meyera; I suspect it's the same adaptive denial at the CDN (which most of us don't have direct visibility into).  Maybe Moritz will comment on what he can see, but I suspect the problem will go away with the denial ages out.

I am seeing this issue as well, and it's still happening after a few days.  It appears to be HTTP version sensitive.

Works, 200:
curl -I -L https://www.php.net/atom.feed --http2

Fail with 503:
curl -I -L https://www.php.net/atom.feed --http1.0
curl -I -L https://www.php.net/atom.feed --http1.1

Guzzle, AFAIK, doesn't support HTTP2, so that means Guzzle cannot fetch the news feed for PHP.net.

Which is... oops.

Hi!

Again a member of the VMware Buildpacks team. We were able to fix the problem by changing the mirror from `https://www.php.net/distributions/php-#{version}.tar.gz` to `https://github.com/php/web-php-distributions/raw/master/php-#{version}.tar.gz` but now we have another problem.

We use this page `https://www.php.net/releases/index.php?json` to get all the information of the releases (SHA, date, if it is a museum version or not, among others) and we start to see the same error previously indicated, all the requests of our automation present a 503 Service Unavailable due to a security check of the CDN.

Error:      	Received unexpected error:
        	            	could not hit php.net: got unsuccessful response: status code: 503, body: <!DOCTYPE html><html><head><title>Security Check</title>

Is there anywhere else I can get this information? In GitHub, there are only the files of each release but not the information of each one of them.

Or if not, is there any way to authenticate/validate the request with a header/token so that this validation does not occur? Currently, this is a big problem for our development process.

I have reached out to the CDN provider to ask for some clarification, and whether we can figure out to stop this from happennig again.

Hi,

Benedikt from Myra Security here, the DDoS-Mitigation Provider from
php.net.

We regularly see attacks on php.net. Sometimes, these attacks share
major details (the fingerprint) with wget.

We just recently have mitigated a wget-based attack.

We removed the rule at around 2022-03-31 23:00 CEST (about 12 hours
ago from now).

So your wget-commands should work again.

Since, this isn't first occurence, we're interally planning to improve
this here in future, that these problems do not persist after the attack
ended.

Thanks for you patience.

@folks of php.net: If you experience problems with the website in the
future, I also think it's ok if you send the e-Mail directly to noc@myra.... (you
should know the rest of the address).

Kind Regards,
Benedikt

Thank you, Benedikt!

Hi Derick and Benedikt,

So far we are still experiencing the problem. We use the native Go library (pkg.go.dev/net/http) to make the request to `https://www.php.net/releases/index.php?json` and keep receiving the 503 Service Unavailable on every request.

Example of failure: https://github.com/paketo-buildpacks/dep-server/runs/5774641790?check_suite_focus=true#step:4:692

Is there any way to sign/authenticate the request to these endpoints? So the CDN does not give us the attack/bot flag.

Thanks!

Hi!

Any update on this issue? We (the VMware Buildpacks team) are still facing these errors when trying to send requests to `https://www.php.net/releases/index.php?json` (receiving a 503 Error and a "Human" Validation)

Moritz Geist from Myra Security here.

We currently have no more blocks in place - the last attack took place about 20 minutes ago. Can you try again?

Thanks Moritz and Benedict for your quick responses.  Hopefully participants on this thread are getting unblocked.

@bhenao - To answer your other question, you can get release info off of github if you're willing to put in a little work.

The source of the data which does into the releases feed you mentioned actually comes from two PHP arrays.  The first can be extracted from https://github.com/php/web-php/blob/master/include/version.inc and represents the current/active releases.

All other (historical) releases are in https://github.com/php/web-php/blob/master/include/releases.inc

Obviously this is a WAY less ideal way to consume the information, but it's there if you want to have a backup for the future.

I like the idea of sending authenticated/token-based requests to bypass automated blocks, though I have no idea if Myra offer that or not.

A third option might just be running your own personal mirror (in a VM!) https://github.com/php/web-php#readme has some quick instructions for setting up a "dev" instance, but if your purposes are about pulling current version information that would probably be sufficient.