# robots.txt for http://www.netassoc.co.nz/ # below disallow all spiders from specified directories user-agent: * user-agent: AhrefsBot disallow: / disallow: /admin/ disallow: /_dev/ disallow: /tmp/ disallow: /private/ disallow: /contact/ user-agent: *