# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file User-agent: * Crawl-delay: 30 Disallow: /account Disallow: /account_link Disallow: /api Disallow: /api_tokens Disallow: /cart Disallow: /cart_link Disallow: /checkout Disallow: /cms Disallow: /orders Disallow: /password Disallow: /user