4 lines
54 B
Plaintext
4 lines
54 B
Plaintext
# block crawling everything
|
|
User-agent: *
|
|
Disallow: /
|
# block crawling everything
|
|
User-agent: *
|
|
Disallow: /
|