21 lines
351 B
Plaintext
21 lines
351 B
Plaintext
|
User-agent: MJ12bot
|
|||
|
Disallow: /
|
|||
|
User-agent: SemrushBot
|
|||
|
Disallow: /
|
|||
|
User-agent: SemrushBot-SA
|
|||
|
Disallow: /
|
|||
|
User-agent: rogerbot
|
|||
|
Disallow:/
|
|||
|
User-agent: dotbot
|
|||
|
Disallow:/
|
|||
|
User-agent: AhrefsBot
|
|||
|
Disallow: /
|
|||
|
User-agent: Alexibot
|
|||
|
Disallow: /
|
|||
|
User-agent: SurveyBot
|
|||
|
Disallow: /
|
|||
|
User-agent: Xenu’s
|
|||
|
Disallow: /
|
|||
|
User-agent: Xenu’s Link Sleuth 1.1c
|
|||
|
Disallow: /
|