User-agent: * Disallow: / Disallow: /email robots Allow index all: InfoSeek Sidewinder Ultraseek Mozilla Lycos_Spider_(T-Rex) Googlebot/1.0 Inktomi Slurp Slurp.so Openfind data gatherer@Openbot flunky crawler_admin@bigfoot.com parallelContextFocusCrawler1.1 larbin_2.2.0 (crawl@compete.com) Bot mailto:craftbot@yahoo.com SpaceBison webbandit BunnySlippers ScoutAbout Ziggy The Clown From Hell!! LinkWalker LexiBot BaiDuSpider DigOut4U KIT Fireball Microsoft URL Control 5.01.4319 Microsoft URL Control 6.00.8862 moget Xenu_s Link Sleuth 1.0r URL Indexer sitecheck.internetseer.com MFC_Tear_Sample Nokia7110 Nutscrape TV33_Mercator_1 1.0 teomaagent crawler admin@teoma.com bumblebee@relevare.com Mitsu CSE@IITBombay MARS SV AtlantisSearch RepoMonkey Bait & Tackle spider.yellopet.com www.yellopet.com The Intraformant SwishSpider SlySearch (slysearch@slysearch.com) AmigaVoyager Toutatis 2.5 2 (http: GaisLab data gatherer, Gaisbot Zeus ThemeSite Viewer Webster Pro V2.9 Win32 WIBBLE WOBBLE Surfnomore Spider v1.1 WebSauger 1.20b UIowaCrawler Kenjin Spider Robot@SuperSnooper.Com 16274.345.67.23 WebWasher 3.0 NOSEYBOLLOCKS V1.0 GB2 LinkChecker EmailWolf 1.00 EmailSiphon