User-agent: Gigabot Disallow: / User-agent: NaverBot Disallow: / User-agent: TurnitinBot Disallow: / User-agent: FAST-WebCrawler Disallow: / User-agent: Mercator Disallow: / User-agent: NPBot Disallow: / User-agent: ZyBorg Disallow: / User-agent: Slurp Disallow: / User-agent: RoboCrawl Disallow: / User-agent: OmniExplorer Disallow: / # Why is your bot excluded? # See http://www.perldesignpatterns.com/robots.txt # Microsoft is offering to pay sites to de-index themselves from # other search engines. Since they're willing to purchase changes # in robots.txt, they can purchase the removal of this directive: User-agent: msnbot Disallow: /