What Is a Sitemap Generator (and When You Need One)

Jul 28, 2026 · SEO Glossary

I once audited a site whose sitemap contained 4,200 URLs. The site had about 300 real pages. The other 3,900 entries were paginated archives, tag pages, print versions, and — my favorite — URLs pointing at the staging subdomain. Nobody had written that sitemap by hand. A plugin generated it, someone ticked every checkbox during setup, and it sat there quietly misleading Googlebot for two years. That's the double edge of the tool this article covers, so let's answer what is a sitemap generator and, more usefully, how to configure one that helps instead of hurts.

What Is a Sitemap Generator, Practically Speaking?

A sitemap generator is any tool that produces an XML sitemap for you — a machine-readable list of the URLs on your site that you want search engines to crawl and index. The output is always the same format:

<url><loc>https://example.com/pricing/</loc><lastmod>2026-07-14</lastmod></url>

What differs is where the URL list comes from, and that's the detail that decides whether your sitemap is trustworthy.

The Three Kinds of Generator

CMS-native generators and plugins

WordPress has shipped a basic sitemap at /wp-sitemap.xml since version 5.5, and plugins like Yoast or Rank Math replace it with configurable versions. Shopify, Wix, and Squarespace generate sitemaps automatically with almost no configuration surface. These generators read from the database, so they know about every post, page, and taxonomy — including ones you'd rather Google never saw. The work here is exclusion: turning off tag archives, author pages, and attachment URLs that add crawl noise.

Crawler-based generators

Desktop tools like Screaming Frog, or online generators, discover URLs by crawling your site the way a bot would, then export the result as XML. The strength is honesty — the sitemap contains only pages actually reachable by links. The weakness is staleness: the file is a snapshot. Publish three posts next week and your sitemap doesn't know. Crawler-based generation suits small brochure sites that rarely change; it's a bad fit for anything with a publishing cadence.

Build-time and framework generators

If you're on Next.js, Astro, Hugo, or any static-site setup, the cleanest approach is generating the sitemap during the build from your route manifest. Packages like next-sitemap or a 30-line script iterating your content collection produce a file that is correct by construction — every deploy regenerates it. This is what I recommend to developer teams, because the sitemap can never drift from reality.

Configuration Mistakes That Poison the Output

• Including noindexed pages. A sitemap says "please index this"; a noindex robots meta tag says "don't". Sending both signals for the same URL wastes crawl requests and clutters Search Console's indexing report with "Excluded by noindex" noise.
• Including redirects and 404s. Every URL should return a clean 200. Generators that read from a database don't know you redirected /old-pricing last month.
• Faked lastmod values. Some plugins stamp every URL with today's date on every regeneration. Google has said it ignores lastmod from sites that abuse it, and once ignored, the field stops helping your genuinely updated pages get recrawled faster. Emit real modification dates or omit the tag.
• Absolute URLs pointing at the wrong host. Staging domains, http:// instead of https://, or bare IPs. This happens when the generator builds URLs from a misconfigured base-URL setting rather than the request host.

How to Verify the Generated File

Don't trust the generator; check its output. Three fast checks:

1. Fetch the sitemap with curl and confirm it returns 200 with a Content-Type of application/xml or text/xml.
2. Sample 20 URLs from the file and confirm each returns 200 — not 301, not 404. An automated audit does this at scale: WebsiteChecker.Tech crawls your sitemap alongside your site and flags every sitemap URL that redirects, errors, or carries a noindex tag.
3. Submit the file in Search Console and watch the "Pages" indexing report for a week. A healthy sitemap shows most URLs moving to "Indexed"; a poisoned one piles up entries under "Page with redirect" and "Not found (404)".

Do You Even Need One?

Honest answer: a 15-page site with clean navigation doesn't gain much, because Googlebot discovers everything through links in one pass. Generators earn their keep when discovery is genuinely hard — sites past a few hundred URLs, weak internal linking, pages rendered client-side, frequent publishing, or news content where minutes matter. They also matter for large sites where crawl budget is a real constraint and you want Google spending requests on canonical pages rather than parameter variations. If that's you, pick the generator closest to your source of truth — database for CMS sites, build pipeline for static sites — configure the exclusions once, and then audit the output on a schedule instead of assuming it stays correct.