User-agent: * Crawl-delay: 60 Request-rate: 1/60 # maximum rate is one page every 60 seconds Visit-time: 0000-2359 # only visit between 00:00 and 23:59 UTC (GMT) Disallow: /Zucchini/* Disallow: /Zucchini/Robots/ Disallow: /*.ico$ Disallow: /*.bmp$ Disallow: /*.gif$ Disallow: /*.jpg$ Disallow: /*.mpeg$ Disallow: /*.rm$ Disallow: /*.wmv$ Disallow: /*.wma$ Disallow: /*.wav$ Disallow: /*.js$ Disallow: /*.xls$ # # Crawln.DoNetDotCom.org #User-agent: dotbot #Disallow: / #