robots.txt Generator
Pick your crawl rules and get a valid robots.txt you can drop straight into your site root.
Default policy
Block specific bots
robots.txt
User-agent: * Disallow: /cart Disallow: /checkout Disallow: /search Disallow: /*?sort=
Save this as robots.txt in your site root, e.g. https://example.com/robots.txt
How it works
Add your sitemap URL, choose which crawlers to address, then add Allow and Disallow rules. The tool builds a valid robots.txt file you can paste into your site root.