User-agent: * Disallow: /archives/*.xml$ Disallow: *ule* Disallow: /archives/2001/ Disallow: /archives/2004/ Disallow: /archives/2005/ Disallow: /answers/ Disallow: /hit/ Disallow: /gg/ Disallow: /scripts/ Disallow: /products/ Disallow: /cgi-bin/ Disallow: /sourcefiles/ Disallow: /test/ Disallow: /wordpress/ Disallow: /atom.xml Disallow: /sitemap.xml Disallow: /index.xml Disallow: /index.rdf Disallow: /rsd.xml Disallow: /events/ # Commenting out the following so that spiders follow them to the # new site via .htaccess redirect # Disallow: /kb/ # Disallow: /gallery/ # Disallow: /news/ # Disallow: /site/ # Disallow: /tag/ # Disallow: /blog/ # Disallow: /pages/