User-Agent: * Disallow: /cgi-bin/ Disallow: /js/ Disallow: /img/ Disallow: *.js Disallow: *.zip Disallow: *.exe Disallow: *.dll Disallow: *.png Disallow: *.gif Disallow: *.jpg User-Agent: ia_archiver Disallow: /