One of our clients published a major update to their site this week. One of the first things I check when what had been a dev site goes live is the robots.txt file, because I want to make sure they didn’t make the mistake of carrying over a file that disallows search engine spiders from crawling the entire site (it’s a pretty common error).
As it turns out, they didn’t make that mistake. They did, however, add a group of three disallow directives that I thought were worthy of note:
The company’s devs added a reference to Asimov’s Three Laws of Robotics. Geek culture FTW.
Overdrive Interactive is an SEO company in Boston that drives measurable ROI. Our search engine optimization programs not only drive natural search traffic – they connect prospects to high quality user experiences that encourage desired behavior. Our SEO services and the resulting search engine presence connects visitors to content that satisfies their needs. As an award-winning Boston SEO company, we are passionate about your success!
