User-agent: ia_archiver User-agent: wget User-agent: webzip User-agent: webmirror User-agent: webcopy User-agent: wwwcopy User-agent: WebReaper User-agent: WebRecorder User-agent: WWWoffle Disallow: / User-agent: * # directed to all spiders, Disallow: /bin/ Disallow: /cgi-bin/ Disallow: /backroom/exclude/ User-agent: ia_archiver Disallow: /