23 lines
385 B
Plaintext
23 lines
385 B
Plaintext
User-agent: MJ12bot
|
||
Disallow: /
|
||
User-agent: SemrushBot
|
||
Disallow: /
|
||
User-agent: SemrushBot-SA
|
||
Disallow: /
|
||
User-agent: rogerbot
|
||
Disallow:/
|
||
User-agent: dotbot
|
||
Disallow:/
|
||
User-agent: AhrefsBot
|
||
Disallow: /
|
||
User-agent: Alexibot
|
||
Disallow: /
|
||
User-agent: SurveyBot
|
||
Disallow: /
|
||
User-agent: Xenu’s
|
||
Disallow: /
|
||
User-agent: Xenu’s Link Sleuth 1.1c
|
||
Disallow: /
|
||
User-agent: AhrefsBot
|
||
Disallow: /
|