Posted By

hotdiggity on 03/30/09


Tagged

robotstxt


Versions (?)

Who likes this?

1 person have marked this snippet as a favorite

AshMan


Robots.txt to hide directory without revealing directory's name


 / Published in: HTML
 

Hides directories from robots without revealing the names of the directories. Robot ignores directory names starting with first two characters.

  1. # robots.txt for http://www.apimagazine.com.au/
  2. User-agent: *
  3. Disallow: /as
  4. Disallow: /ca
  5. Disallow: /cg
  6. Disallow: /do
  7. Disallow: /la
  8. Disallow: /ma
  9. Disallow: /op
  10. Disallow: /st
  11. Disallow: /te

Report this snippet  

You need to login to post a comment.