Skip to Main Content


Robots.txt File

By: Lee Johnson - SEO Manager - CustomerStreet
Comments ( 0)
Oct
23

Place a robots.txt file on every domain. The first thing a spider looks for when it hits your Web site is a robots.txt file. If it’s looking for that file, put it on your site! Robot Manager Pro is an excellent tool for creating robots.txt files as well as monitoring spider traffic. Download a free copy at websitemanagementtools.com.

If you liked that, then try ...

Capitalisation of Words
This article is now defunked as Google has changed its algorithm.

What is Latent Semantic Indexing (LSI)?
What is LSI? With all of our Customer Street Industry specific directories we keep LSI in mind.

  • Digg
  • Technorati
  • del.icio.us
  • BlinkList
  • Reddit
  • StumbleUpon
  • Furl
  • YahooMyWeb
1 Star2 Stars3 Stars4 Stars5 Stars
2.5
after 2 Votes

Leave a comment

Back to Top