Last week, I went to blocklayer to create an xml sitemap for Google. I first submitted SEO Blackhat’s xml site map on August 20th and was deep crawled 6 hours later.
Go here to submit your Google Sitemap.
I updated the xml Google sitemap early yesterday and again resubmitted. Within an hour, the Googlebot was slurping away: 20 Mbs later, Google had cached every page (all 168 I believe) on the sitemap.
You can see which pages are cached in Google with the SEOpen firefox Extension.
The place to submit a sitemap to Yahoo is here. You can provide the location of a text file containing a list of URLs, one URL per line, called sitemap.txt.. This is the text SEO Blackhat sitemap.
Sure the Inktomi Slurp bot (Yahoo) came by yesterday, but it indexed much less than the Googlebot. The sitemap submission to Yahoo does not seem to have done anything . . . yet.
Google probably would not have been so responsive if SEO Black Hat did not have so many high quality backlinks (even though Google’s worthless link checker claims to see none of SEO Blackhat 100 or so backlinks).
Grey Wolf did a more controlled Google Sitemap test by pointing just one PR 4 at each site in his experiment.
The sites that did not use the Google sitemap had their homepages indexed in approximately the same time frame, so there was no benefit in the time to index. However each of the sites using Sitemap were deep crawled and deep indexed. Yes 100% deep crawled and 100% indexed, in some cases hundreds of files.
For Black Hat sites, I do not recommend submitting site maps of larger than 750 pages at a time. In the past, when we Blog & Ping, we sent out links to mini HTML site maps with 250 internal links. This allowed us to ramp what we show to the search engines over time and stay under the spam radar.
If you try to drop a 30,000 pages site map of a new site on Google or Yahoo, you’re asking for trouble and you’ll probably get it.
Over the next few weeks, we will be experimenting with mini Google sitemaps on our black hat search engine spam sites. We will compartmentalize the sites into blocks of 500 pages or so and experiment with different submission and B&P rates. We will keep you posted on the results.
ok … i finally got around to doing this. i registered a domain last week, created an article bot project, created an XML sitemap with this gem - http://www.vigos.com/products/gsitemap/ and jsut submitted the project to G.
let’s see.