User-agent: * Disallow: /cgi-bin/ Disallow: /comment Disallow: /contact/ Disallow: /dev/ Disallow: /esu/unitprogcat/ Disallow: /eopro/ Disallow: /feedback/ Disallow: /fluff/ Disallow: /imagelib/ Disallow: /logs/ Disallow: /news/news-results.html Disallow: /news/news-search.html? Disallow: /oenology Disallow: /prospectus/postgraduate/2014/ Disallow: /prospectus/undergraduate/2013/ Disallow: /slink/ Disallow: /sso/login Disallow: /test/ Disallow: /webcam/sport/archive/ Disallow: /biology/people/person/ Disallow: /composites/media/ICCM17/ Disallow: /composites/media/icons/ Disallow: /composites/media/icshm/ Disallow: /Control_Panel/ Disallow: /Depts/CAS/ Disallow: /depts/CAS/ Disallow: /geography/people/person/ Disallow: /history/customs/resources/ Disallow: /history/customs/datasets/ Disallow: /is/cgi-bin/ Disallow: /is/comment Disallow: /research/funding/salary_costing/ # AFAIK only Googlebot understands these Disallow: /*/comment$ Disallow: /*/comment?url Disallow: /*/feedback$ Disallow: /*/news-results.html Disallow: /*/news-search.html? Disallow: /*/safe/