Enter your website address
Used for the sitemap line and to resolve page paths.
Free tool · Robots.txt & sitemap
Create a robots.txt file and an XML sitemap in a minute, with sensible rules for WordPress. Choose a preset, adjust the rules, then copy or download the file. It runs in your browser, and nothing you type is sent anywhere.
Used for the Sitemap line. https:// is added if missing and any path is removed.
One per line. A leading / is added if missing.
Google ignores Crawl-delay. Some other crawlers, such as Bing, respect it.
Tick a crawler to block it with its own Disallow: / group. Blocking AI crawlers may reduce how often your content appears in AI answers; whether to block is your call.
Upload to your site root so it loads at yourdomain.com/robots.txt. Many SEO plugins edit this file under SEO → Tools.
Optional. Lets you enter relative paths such as /about/.
One per line. Blanks are ignored and duplicates are removed.
Leave empty to omit priority.
Google ignores priority and changefreq. On WordPress, an SEO plugin generates your sitemap automatically, so this tool suits static or small sites, or checking structure. Submit the finished file in Google Search Console.
Runs in your browser. Nothing you enter is uploaded or stored.
Used for the sitemap line and to resolve page paths.
WordPress, allow everything, block everything or custom.
Add paths to block or allow, and choose any crawlers to block.
Upload the file to your site root, or enter the rules in your SEO plugin.
Straight answers about this tool. If yours isn't here, ask me directly.
It is a text file at the root of your site that tells crawlers which parts they may visit. It guides crawling, but it is not a security tool.
It blocks the admin area, keeps admin-ajax.php available, blocks internal search pages and adds your sitemap line.
Not always. A blocked page can still be indexed if other sites link to it. Use a noindex tag to keep a page out of results.
That is your call. Blocking them can stop your content being used for training, and may reduce how often you appear in AI answers.
No. Plugins such as Yoast and Rank Math create one automatically. This tool is useful for small or static sites.
No. The files are built in your browser.
The hosting I use
In my experience it's one of the best hosting options for small business websites, and it's the one I suggest. If you sign up through my link, I earn a small commission at no extra cost to you.
These tools are free to use. If your site needs more than a new icon or better markup, I design and build custom websites for small businesses, and I offer a free audit that shows you what to fix first.