← Back to CheckerLeaderboardDashboard
Tool · llms.txt Generator

Generate an llms.txt for your website

Paste your homepage URL. We'll crawl your site, pick the most useful pages, and produce a properly formatted llms.txt you can drop on your server. Free with a Crawl Readiness account — sign up takes 30 seconds.

What is llms.txt?

llms.txt is a proposed standard (like robots.txt or sitemap.xml) that gives AI assistants a curated map of your site's most useful pages, plus short descriptions of what each one covers. When ChatGPT, Claude, or Perplexity encounter your site, they can read llms.txt first to understand what you offer before crawling individual pages.

Why it matters

  • AI assistants get a cleaner, faster read of your site
  • You control which pages get highlighted
  • Descriptions help AI cite you accurately
  • It sits alongside your existing sitemap and robots.txt

How to use the output

  1. Generate the file with the form above
  2. Edit the descriptions to match your voice
  3. Upload it to yourdomain.com/llms.txt
  4. Add the companion lines to your robots.txt

Not sure where to put the file? Pick your platform

Every platform is a little different. Click the one your site runs on for step-by-step instructions.

WordPress

Uploading llms.txt

  1. In your WordPress admin, install the free plugin "WPCode" (or "Insert Headers and Footers") — Plugins → Add New → search for it.
  2. This won't help you upload files though — for a text file at the site root you need FTP or your host's file manager. Log in to your hosting provider (Bluehost, SiteGround, etc.) and open File Manager.
  3. Navigate to the public_html folder (or whatever folder your WordPress lives in — the one containing wp-config.php).
  4. Upload the llms.txt file you downloaded straight into that folder.
  5. Confirm it works by visiting yourdomain.com/llms.txt — you should see the text you generated.

Updating robots.txt

  1. In WordPress admin, if you use Yoast SEO or Rank Math, go to that plugin's Tools → File editor (Yoast) or General → Edit robots.txt (Rank Math).
  2. Paste the companion robots.txt lines at the bottom of the existing file — do NOT delete what's already there.
  3. Save. If you don't have those plugins, edit the robots.txt file via File Manager the same way you uploaded llms.txt.
Shopify

Uploading llms.txt

  1. Shopify doesn't let you upload arbitrary files to your site root through the admin. You'll need to use the Files section as a workaround — but files there sit at cdn.shopify.com URLs, not yourdomain.com/llms.txt.
  2. Best option: install a free Shopify app called "EasyRoutes" or "File Uploader Pro" that lets you serve custom files at a chosen path. Search the Shopify App Store.
  3. Alternative: if you have a Shopify Plus plan, you can use Shopify's built-in Files feature with a custom domain proxy. Contact Shopify support for guidance.
  4. Once installed, upload llms.txt via the app's interface and set the path to /llms.txt.
  5. Verify at yourdomain.com/llms.txt.

Updating robots.txt

  1. Shopify auto-generates robots.txt, but you can customize it. In your Shopify admin, go to Online Store → Themes → Edit code.
  2. In the Templates folder, click "Add a new template" → choose robots → Create.
  3. This opens robots.txt.liquid. Add the companion lines inside a {% for group in robots.default_groups %} loop or as raw text at the bottom.
  4. Save. Shopify will now serve your customized robots.txt.
  5. Full docs: help.shopify.com — search "customize robots.txt".
Squarespace

Uploading llms.txt

  1. Squarespace does not currently support uploading raw text files like llms.txt to your site root — this is a known limitation.
  2. Workaround #1: create a new blank page at /llms — Pages → + → Blank Page. Set the URL slug to "llms". Add a single Code block with your llms.txt content wrapped in <pre>...</pre>. The URL won't end in .txt but AI crawlers can still find useful content there.
  3. Workaround #2 (better): use a service like GitHub Pages or Netlify to host just the /llms.txt file, then set a redirect from yourdomain.com/llms.txt to it via Squarespace's URL Redirects (Settings → Advanced → URL Mappings).
  4. For most Squarespace sites, the JSON-LD structured data (coming soon as a separate tool) will give AI crawlers more usable info than llms.txt alone.

Updating robots.txt

  1. Squarespace does NOT allow editing robots.txt directly. It's auto-generated and can't be customized on most plans.
  2. The good news: Squarespace's default robots.txt already allows most AI crawlers.
  3. If you need custom rules, contact Squarespace support — they can sometimes make exceptions for business/commerce plans.
Wix

Uploading llms.txt

  1. Wix does not allow uploading arbitrary text files to your site root through the standard editor.
  2. Workaround: create a hidden page at /llms with a single text element containing the llms.txt content. This won't be at a .txt URL but AI crawlers may still index it.
  3. Best solution: upgrade to a Wix Business or Enterprise plan and use their Velo developer platform — you can serve a custom endpoint at /_functions/llms that returns your content with the right headers.
  4. Contact Wix Customer Care if you're on a lower plan and this is important — they occasionally help with custom setups.

Updating robots.txt

  1. Wix generates robots.txt automatically and doesn't allow direct editing on Free, Combo, or Unlimited plans.
  2. On Business plans, you can override it via Wix's SEO settings → Robots.txt Editor.
  3. Paste the companion lines at the bottom of whatever's already there, save, and publish.
Webflow

Uploading llms.txt

  1. Webflow doesn't have a native "upload a text file" feature, but there's a clean workaround.
  2. In your Webflow project, go to Site Settings → SEO tab → scroll to "robots.txt" (we'll handle llms.txt differently).
  3. For llms.txt itself: create a new blank page named "llms" with URL slug "llms.txt" — Webflow will complain about the .txt extension, so use "llms" instead and serve it as HTML.
  4. Best solution: host llms.txt on a separate service (like a plain S3 bucket or GitHub Pages) and set a 301 redirect from yourdomain.com/llms.txt → that URL. Webflow → Site Settings → Publishing → 301 Redirects.
  5. Or use Webflow's Cloudflare integration if you have it — Cloudflare Workers can serve arbitrary files at any path.

Updating robots.txt

  1. In Webflow, go to Site Settings → SEO tab.
  2. Scroll down to the "robots.txt" field.
  3. Paste the companion robots.txt lines at the bottom — don't delete anything Webflow put there by default.
  4. Save changes and republish your site.
Vercel / Netlify

Uploading llms.txt

  1. Put your llms.txt file in the /public folder of your project (Vercel/Next.js) or /static folder (some frameworks).
  2. Commit and push to Git. Vercel/Netlify will auto-deploy.
  3. The file will now be served at yourdomain.com/llms.txt automatically.
  4. That's it — no config needed. Anything in /public is served at the site root.

Updating robots.txt

  1. Same as llms.txt: drop robots.txt (or add to your existing one) in the /public folder.
  2. If you already have a robots.txt there, open it in your editor and paste the companion lines at the bottom.
  3. Commit, push, redeploy. Done.
  4. For Next.js apps that generate robots.txt dynamically, add the rules to your robots.ts or next-sitemap config instead.
Cloudflare Pages

Uploading llms.txt

  1. Add llms.txt to the root of your Pages project (or /public if you use a framework like Next.js).
  2. Commit and push to your Git repository — Cloudflare Pages will build and deploy automatically.
  3. For static-only sites, you can also use Cloudflare's dashboard: Pages → your project → Files (if enabled) to upload directly.
  4. Verify at yourdomain.com/llms.txt.
  5. Bonus: use a Cloudflare Worker to serve llms.txt from any URL if you need more control — search "Cloudflare Worker static file" for a snippet.

Updating robots.txt

  1. Same process: put robots.txt in the root or /public folder of your project.
  2. If your framework already has one, paste the companion lines at the bottom.
  3. Commit and redeploy.
cPanel / FTP (any host)

Uploading llms.txt

  1. Log in to your hosting control panel (usually cPanel, Plesk, or your host's custom dashboard).
  2. Open the File Manager tool.
  3. Navigate to the public_html folder (or www, or the folder your website lives in — it's the one containing index.html or index.php).
  4. Click Upload and select the llms.txt file you downloaded.
  5. Once uploaded, verify by visiting yourdomain.com/llms.txt in a browser — you should see the text.
  6. If you prefer FTP: use FileZilla or Cyberduck, connect with your host's FTP credentials, and drop llms.txt into the same folder.

Updating robots.txt

  1. In the same File Manager (or FTP client), find the existing robots.txt in your public_html folder.
  2. If it exists, right-click → Edit. Paste the companion lines at the very bottom, save, close.
  3. If it doesn't exist, upload the robots-addendum.txt file you downloaded and rename it to robots.txt.
  4. Verify at yourdomain.com/robots.txt.

Stuck? Email crawlreadiness@gmail.com with your platform name and we'll walk you through it.