Robots.txt Generator

Generate a robots.txt file with rules for crawlers, sitemaps, and disallow rules.