code snippets
all code snippets
/
popular code snippets
/
your code snippets
extras
blog
privacy
about snipplr
Change style:
Big
/
Small
New Snippet
/
Register
/
Login
advanced code snippet search
login /
Register
Username:
password:
Login with
Facebook
Share Snipplr
Tools
Textmate Bundle
WordPress Plugin
Snippets
More tools ...
Popular Tags
AVG
Centers
Chennai
Delhi
Desk
Development
Green
HTAcess
Health
Mcafee
Park
Plesk
Spa
VirtualHost
antivirus
apache
authentication
block
body
cache
codeigniter
compress
config
contact
css
custom
customer
domain
domains
download
drupal
dynamic
embedded
error
expire
font-face
force
gzip
header
help
helpline
host
hotlinking
htaccess
html
http
https
images
in
ip
joomla
js
linux
maintenance
massage
mobile
modrewrite
module
number
openbasedir
parlour
password
performance
phone
php
project
proxy
redirect
redirection
regex
remove
rewrite
rewriting
robotstxt
security
seo
server
service
site
slashes
ssl
static
subversion
support
svn
technical
uk
url
urls
vhost
virtual
wordpress
www
zip
301
403
404
500
Everyone's Recent Apache Snippets Tagged 'robots'
All
/
JavaScript
/
HTML
/
PHP
/
CSS
/
Ruby
/
Objective C
« Prev [Page 1 of 1] Next »
Simple Robots.txt Examples
Apache
server
apache
admin
robots
posted on June 18, 2011 by
ryarwood
« Prev [Page 1 of 1] Next »
Choose a language for easy browsing:
ActionScript
ActionScript 3
Apache
AppleScript
ASP
Assembler
AutoIt
Awk
Bash
C
C#
C++
Clojure
ColdFusion
CSS
Delphi
Diff
Django
DOS Batch
Emacs Lisp
eZ Publish
Forth
Fortran
Gnuplot
Groovy
HAML
Haskell
HTML
iPhone
Java
JavaScript
jQuery
LaTeX
lighttpd
Lisp
Lua
Makefile
MatLab
Maxscript
Mel
MXML
MySQL
NewtonScript
Objective C
Open Firmware
Other
Pascal
Perl
PHP
PicBasic
PL/SQL
Processing
Prolog
Pseudocode
Python
R
Rails
Regular Expression
Revolution
Ruby
SAS
SASS
Scala
Scheme
SmallTalk
Smarty
SML
SPSS
SQL
SVN
Symfony
TCL
Textpattern
TYPO3
VB.NET
VHDL
Visual Basic
W-Language
Windows PowerShell
Windows Registry
XHTML
XML
XSLT