php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #24901 Remove mirrors of PHP website from google to make finding solutions easier
Submitted: 2003-08-01 06:35 UTC Modified: 2004-02-08 05:50 UTC
From: webmaster at geog dot cam dot ac dot uk Assigned:
Status: Not a bug Package: Website problem
PHP Version: Irrelevant OS: n/a
Private report: No CVE-ID: None
View Add Comment Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
You can add a comment by following this link or if you reported this bug, you can edit this bug over here.
(description)
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: webmaster at geog dot cam dot ac dot uk
New email:
PHP Version: OS:

 

 [2003-08-01 06:35 UTC] webmaster at geog dot cam dot ac dot uk 
Description:
------------
Can consideration be given to putting a robots.txt containing

User-agent: *
Disallow: /

for mirrors of the PHP website? I.e. only php.net would not disallow google in.

It is very annoying when using google/whatever to find a solution to a problem to be confronted with continual multiple copies of the same thing.

Given that the main www.php.net has redirections to local mirrors installed, google searching will take people to the local mirror anyway.

Presumably this would take a while to filter through to all the mirrors.

There might be an issue with using site:[mirrorname] [searchterms] but this would be outweighed by not having to plough through multiple copies (many of which are out of date) of the PHP documentation.

Patches

Add a Patch

Pull Requests

Add a Pull Request

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2003-08-01 06:40 UTC] goba@php.net 
We have discussed this point several times, and the outcome was always the same. We would not like to disallow mirrors to be indexed. If you would like to restrict the search to www.php.net only, you can do it on Google, and many other search sites. Our current site search excatly does this.
 [2003-08-01 06:44 UTC] webmaster at geog dot cam dot ac dot uk 
> We have discussed this point several times, and the
> outcome was always the same

But might this be worth reconsidering now that the redirection system is in place?

> If you would like to restrict the search to
> www.php.net only

No, the problem is when you're trying to find results that are _not_ from the PHP documentation, i.e. when the documentation doesn't solve the problem.
 [2003-08-01 07:18 UTC] hholzgra@php.net 
had the very same problem yesterday ...

pdf_setdash() documentation is lousy and i tried to find some example script that uses it on google 

no way!

and it is not only the official mirrors, i also got lots of hits for copies of the manual pages on other servers

even if we want the official mirrors to be indexed (i seem to have missed these discussions so i don't know the pros and cons) we should IMHO add an option to generate noindex meta tags for the HTML formats of the manual

this should be controlable using configure and enabled by default

manuals for php.net would then be created with the option explicitly turned off (with the option of overriding it in robots.txt on mirrors)
 [2003-08-01 07:33 UTC] goba@php.net 
OK, so let the generated files be disallowed for robots. I have added a meta tag for printed pages immediately, so those will not be indexed anymore (this cannot be expressed in robots.txt AFAIK)...
 [2004-02-07 14:03 UTC] nlopess@php.net 
This is a website problem.

I think thats a good idea to disallow crawlers to index mirrors. it becomes a messy when you want to find something....
 [2004-02-08 05:50 UTC] goba@php.net 
People should look a bit deeper into Google. It is not just one input line... See the advanced search page. You can restrict a search to only the www.php.net site in case you are not interested in mirrors, or you can disable searching on all php.net subsites in case you would like to get some pages, examples, articles, etc. from different servers...
 
PHP Copyright © 2001-2024 The PHP Group
All rights reserved. 		Last updated: Sun Apr 28 06:01:30 2024 UTC