Free Tool from DreamLaunch
Robots.txt Generator
Block or allow specific crawlers, including AI bots, line by line.
Free, no signup. Updates live as you type.
robots.txt
User-agent: * Disallow: /admin Disallow: /api User-agent: Googlebot Disallow: User-agent: Bingbot Disallow: User-agent: GPTBot Disallow: User-agent: ClaudeBot Disallow: User-agent: PerplexityBot Disallow: User-agent: Google-Extended Disallow:
Frequently Asked Questions
Everything to know about robots.txt.
Save it as robots.txt in your site's root directory so it's reachable at yoursite.com/robots.txt.
Our Work
Mizu AI
Shipping an AI-native automation builder from zero to launch in 6 weeks
Product DesignFull-Stack Build
Mrsam AI
The RTL-first, no-code website builder that closed a $500K seed round
Product DesignAI Integration
Aprex
From blank canvas to production app: a precision tool for thought in React
ReactWeb App
Book a Call