RSS Feed
Twitter
November 25th, 2005 
QuadsZilla I realize I’m a little late on covering this one.
But honestly, it’s because I just don’t get it.
Brett Tabke of webmasterworld world decided to ban all spiders and remove all his pages from Google and Yahoo. He has been successful at removing all 2 million plus pages from Google’s index.
Usually I can explain why things happen – it’s something I’m good at. Today I’m stumped.
Since I doubt anyone will have the real answer to this mystery, I want to open it up to you for wild and absurd speculation.
After all:
“Once you rule out the impossible, whatever remains, however improbable, must be the truth” -Sherlock Homes
So if you’re game, give me your unlikely but possible explanation for why Brett did this. (Suggested themes: blackmail, revenge, conspiracy, doppelgangers, government, FSM, prank . . . or if you got it the TRUTH).
Posted in 
WMW has been banned. No way to remove it by yourself so fast.
The answer comes down to bots. Not bots that obey robots.txt, but other “bad” bots. These undesirable bots have been sucking up huge amounts of bandwidth. Webmasterworld now requires cookies and has taken some other anti-bot measures, and they felt that cloaking for Google to allow continued spidering by G-bot would not be in compliance with Google’s quality guidelines. Hence, the complete exclusion of bots, good and bad.
well, me for example, was reading webmasterworld using google index, instead of paid subscription =)
it was set up that if you have a “Referer:” as google SERPS page, you can view that single page of interest (as they needed to attract new members somehow).
I think that the answers are in the articles about this. The problem is that the search engine bots took up to mauch bandwith and the costs were increasing, and on the other side WW has enough popularity in order to not care about search angines traffic.
“one should not increase, beyond what is necessary, the number of entities required
to explain anything.”
Ocam’s Razor
I guess it just might be an ego thing – I think he wants to prove he can survive without SE refferals just by being a valuable resource to the community.
Here’s what I think. There are lots of big web sites who manage to cope with bots, good and bad. The cult of WmW ain’t special in that regard. If it can’t cope with that then it smacks of massively ironical incompetence. Couple this with self importance and a case supposedly emerges to ban all bots, require cookies etc. Like that’s gonna help.
I think the site owner wants to monetise the site, reckons that the members won’t put up with ads on all the pages, isn’t imaginative enough to only shows ads to non-members and thinks that a site search within site control would afford that opportunity.
I’ve been an occasional poster and regular reader at WmW for a few years; as of a few days ago the site is next to useless. This has provided the impetus to read around a little more; and the searchable forums (without control freak rules on how to post right, how to read right, how to breathe right) are going to be the winners in this.
I agree with Thai Wun Sen. Any other explanation is just spin to save face. They had tons of mirror sites an it caught up with them.
Maybe they got hacked Muahahahaha!
1) You can’t remove pages that fast from google index. There will always be some entries without discription – even if the sites don’t exist anymore.
2) WMW cloaked via HTTP_REFERRER.
=> WMW has been banned.
While it seems impressive that one can run an online business by denying free traffic from Google, as a business person I wouldn’t do it. I always thought bandwidth was cheap and I still believe so.