Tool
Verse
All Tools
SEO
Image
Text
Dev
Calculators
Converters
Home
SEO Tools
Robots.txt Generator
Robots.txt Generator
Create a proper robots.txt file to control search engine crawler access.
Default Crawlers Rule
Allow all crawlers
Block all crawlers
Disallow Paths (one per line)
Allow Paths (one per line)
Block Specific Crawlers
ChatGPT (GPTBot)
Common Crawl
Claude (Anthropic)
Google Bard
Facebook
Twitter/X
Ahrefs
Semrush
Majestic
Moz
Sitemap URL (optional)
Generated robots.txt
Copy
Download
Popular Tools
Backlink Checker
Regex Tester
Word Counter
Plagiarism Checker
Text Diff Checker
Meta Tag Generator
Categories
Text Tools
SEO Tools
Image Tools
Developer Tools
Calculator Tools
Converter Tools