Sitemap Generator

🌐 Sitemap Generator

Purpose

Generate or download an XML sitemap for your shop’s website. The Sitemap Generator automatically tries to locate an existing sitemap (robots.txt → sitemap.xml). If none is found, it performs a limited crawl of your shop’s website and creates a new sitemap XML file, stored securely on the server for download.

🧭 What this feature shows

Sitemap History — a timestamped list of all sitemap generations, showing:

  • Performed date/time

  • Website URL

  • Number of URLs included

  • Generated sitemap file name

  • View/download actions

Sitemap Details — open any sitemap to inspect:

  • Website scanned

  • Number of URLs included

  • Source type (robots.txt, sitemap.xml, or crawled)

  • Summary and notes

  • Sample URLs (first 200 links)

  • Any errors or skipped URLs

Files — Each sitemap is stored on the server (/storage/seo_sitemaps/) and can be downloaded anytime.

👥 Who should use this

  • Local business owners who need a sitemap for SEO indexing or search engine submissions.

  • Marketing/SEO teams validating correct indexing of product or service pages.

  • Developers auditing website link structure or crawl coverage.

⚙️ How it works (high level)

  1. Try auto-discovery The generator first checks for a declared sitemap in robots.txt. If found, it fetches and stores that sitemap.

  2. Fallback to sitemap.xml If robots.txt doesn’t list any, it checks /sitemap.xml directly.

  3. Limited crawl (backup mode) If neither is found, it crawls your domain starting from the homepage — following internal links up to a safe limit — and builds a fresh sitemap.

  4. Generate XML & Store The sitemap is saved in /storage/app/public/seo_sitemaps/ and recorded in the database with metadata (website, URL count, errors, etc.).

  5. View / Download Admins can view sitemap details or download the .xml file for submission to Google/Bing or SEO consultants.

🪜 How to use — step-by-step

  1. Open Dashboard → Sitemaps.

  2. Click Generate Sitemap.

  3. The system checks existing sitemaps or crawls your shop.

  4. Wait for completion — the new sitemap appears in the history list.

  5. Click View to see details:

    • Number of URLs included

    • Sitemap source type (robots/sitemap.xml/crawled)

    • Summary

    • Errors (if any)

  6. Click Download to get the XML file.

  7. Submit it to Google Search Console or Bing Webmaster Tools for improved crawling.

📊 How to interpret results

Field
Meaning

Performed At

Date/time when sitemap generation occurred.

Website

The base domain used for discovery or crawling.

URLs

Total number of unique URLs found and added.

File

The generated XML file (downloadable).

Source

How the sitemap was found (robots.txt → sitemap.xml → crawled).

Summary

Quick text summary (e.g., “Found sitemap via robots.txt”).

Errors / Notes

Any blocked URLs, timeouts, or parse errors during crawling.

✅ Recommended actions (prioritised)

  1. Verify sitemap URLs Ensure key pages (home, products, services, contact) appear in the sitemap.

  2. Submit to search engines

    • Google: via Search Console → Indexing → Sitemaps

    • Bing: via Bing Webmaster Tools → Sitemaps

  3. Monitor URL count Sudden drops in URL count may indicate broken links or missing pages.

  4. Fix reported errors Review “Errors/Notes” for blocked pages or crawl issues.

  5. Regenerate after major updates Re-run sitemap generation after new content, redesigns, or domain structure changes.

📤 Exporting & Reporting

  • Download XML — from the sitemap list or detail page.

  • Shareable Report — you can share the XML file directly with SEO consultants.

  • Manual backup — copy or version-control the XML files from /storage/app/public/seo_sitemaps/.

🗓️ Recommended cadence

Situation
When to run

After major site updates

Immediately after deployment

Routine maintenance

Monthly

SEO audits

Quarterly

Domain change or migration

Immediately after DNS switch

🔒 Privacy & safety notes

  • Only publicly accessible URLs are crawled.

  • The generator does not access private or restricted content.

  • Stored XML files contain only URLs, no user data.

🧩 Troubleshooting & FAQs

Q: Sitemap generation shows “0 URLs”. A: The site may block crawlers via robots.txt, or pages use JavaScript links not visible to basic crawlers. Try allowing bots or manually verifying your sitemap.xml.

Q: File not found when downloading? A: The sitemap may have been deleted from storage. Re-run the generator to create a new file.

Q: Crawl stopped early? A: The crawler has a safety limit to prevent infinite loops. Only top-level internal links are collected.

Q: Sitemap already exists on site? A: The generator will reuse and store the discovered sitemap if found via robots.txt or /sitemap.xml.

✅ Quick checklist

PreviousBacklinksNextInternet Speed Test

Last updated