Hi!, If you're new here, you may want to subscribe to my RSS feed. Thanks for visiting!
I read this article by John Chow on reducing the number of pages in Google’s Supplemental Index and checked out the same for my blog and found that it had more than 700 pages in Google’s Supplemental Index and decided that I would do something about it. I had earlier created a robots.txt based on Everton’s and like him, I also experienced a steep increase in traffic. When a further change guaranteed some more increase in traffic, I was more than happy to try it out.
Here is the SEO for Firefox stats for my blog. This was taken yesterday and there has been absolutely no change since May 24th (That’s when I had changed the robots.txt file).
Based on John Chow’s robots.txt I modified mine. You can see it here. That is when trouble started. My traffic started going down. Drastically. I tried to reason out why it happened. My blogging pattern hadn’t changed. Apart from the robots.txt, I couldn’t find any other difference. When the drop went down by 60%, I reverted back to my old robots.txt (day before yesterday) and immediately I began to see a slight increase in the number of hits. I also checked out SEO for Firefox for John Chow which shows that his blog has crossed more than 2000 pages in Google’s Supplemental Index. Check out the screenshot below.
This experiment doesn’t seem to have helped John Chow. I’m not sure if he noticed any spike in traffic. It definitely has not helped me in any way. However, Nathan at Not So Boring Life seems to have had an increase of 20% in Search Engine traffic.
Below is an excerpt of my conversation with an ex-colleague who has some experience in SEO
me: then how do i reduce the suplementary indexing?
Anil: Actually once the page rank of ur blog increases, all the pages which are in supplementary index of search engines will come in to main indexing…. no need to worry abt it…. but remove all “disallows” in the code…this will cause more pain than supplementary indexing..
I’m not sure how correct this is, but that’s exactly what I did by going back to my earlier robots.txt. Can someone help me out here? These questions have been bothering me for quite some time now.
- What went wrong with this experiment?
- Why did I lose more than 60% of my traffic?
- Why has the number of pages in Google’s Supplemental Index for John Chow increased while it should have decreased?
- How did it work for Nathan and not for me?
Any answers??
Popularity: 1% [?]
You Should Also Check Out This Post:
- Dongles And Their Usage In Mobile Broadband
- Browser Market Share - After Chrome Day 2
- Solution To Google Chrome Mouse Scroll Issue
- [Free Offer] Download DoubleSafety For Free Only Today!
- Browser Market Share - Before and After Google Chrome
More Active Posts:
- Joost Invites, Anyone? (52)
- Is There A Problem With Using The Scroll Mouse With Google Chrome? (43)
- Popular Desktop Applications And Their Online Alternatives (42)
- Top 10 List of Third Party Web based Instant Messengers (14)
- 23 Vibrant & Colorful Wordpress Themes! (11)
- Lyrics Plugin for Winamp and Windows Media Player (10)
- No Filename/Foldername in Thumbnail View (9)
- How To Delete A Hotmail Or Windows Live Account (9)
- NetAudioAds - Get Paid For Every Visitor To Your Site (9)
- What went Wrong with My Robots.txt? (8)



Not sure why you are seeing a drop in traffic. If your robots.txt file is messed up you’d have seen a ton of your pages become uncached.
Changing your robots.txt file won’t effect your site overnight. You have to wait for the googlebot to recache your entire site which could take 2-3 weeks.