XML Sitemap Audit: How to Catch Missing URLs, Bad lastmod, and Crawl Waste
An XML sitemap audit checks whether your sitemap is acting like a useful discovery map or just sitting on the site as a stale technical artifact.
A sitemap does not need to be perfect to be helpful, but it does need to be trustworthy. If it points to broken URLs, omits important pages, or carries bad freshness hints, it becomes harder for crawlers to use confidently. Google's sitemap guidance points to these rules:
- Google asks for fully-qualified URLs.
- Google limits a single sitemap to 50 MB uncompressed or 50,000 URLs.
- Google says it uses
lastmodonly if the value is consistently and verifiably accurate. - Google ignores
priorityandchangefreq.
What a Sitemap Audit Should Validate
A strong audit should answer a few practical questions:
- Was the sitemap fetched successfully?
- Is the XML valid?
- How many URLs are listed?
- Are there duplicate or invalid URLs?
- Are important crawled pages missing from the sitemap?
- Are lastmod values missing or invalid?
- Are listed URLs outside the intended site scope? A sitemap should list only URLs from the site it belongs to.
- Does the sitemap still reference URLs that return 404?
That list matters because sitemap quality is really about discovery trust. A crawler should be able to treat the file as a reliable hint, not a cleanup project.
Why This Still Matters for AI and Search Agents
Sitemaps are not glamorous, but they can help large and changing sites tell crawlers where the important content lives and when it was last updated. When combined with hreflang validation, sitemap audits give a complete picture of discovery and localization health.
If the sitemap is stale, bloated, or inconsistent, crawlers that rely on it have to work harder to discover the right URLs. Running a full technical readiness audit reveals whether sitemap problems are compounding with other crawl issues.
What AEOprobe Shows in the Sitemap Report
AEOprobe audits sitemap quality as part of the larger AI-search readiness workflow. AEOprobe looks for the sitemap declared in robots.txt, or /sitemap.xml if none is declared. The report surfaces the sitemap conditions most likely to create wasted crawling or weak discovery signals:
- Whether the sitemap was fetched and whether the XML parsed cleanly
- Total listed URLs for scale context
- Duplicate and invalid URL counts
- URLs missing lastmod and URLs with invalid lastmod
- Out-of-scope URLs that point to a different host than the audited site
- Crawled pages missing from the sitemap
- Sitemap entries returning 404
- Parse errors that point to file-level problems
This matters because it turns “the sitemap exists” into a more useful question: “is the sitemap helping or hurting discovery?”
What to Fix First
When the sitemap looks messy, fix the most trust-damaging problems first:
- Repair fetch and parsing failures.
- Remove broken, duplicate, and out-of-scope URLs.
- Add important pages that were crawled but not listed.
- Clean up lastmod data so it reflects the last significant update to each page.
After that, review generation rules. Most sitemap issues are not hand-edit problems. They are template or pipeline problems that keep emitting low-quality entries.
What a Healthy Sitemap Looks Like
A healthy sitemap is boring in exactly the right way. It fetches cleanly, lists the right URLs and stays in scope. Its freshness hints reflect reality.
A healthy sitemap does not try to represent every utility URL on the site. It does not keep obsolete pages hanging around for months.
That kind of cleanliness saves crawl effort and makes the rest of the technical stack easier to trust.
Audit the Sitemap Before It Becomes Background Noise
Teams often discover sitemap problems late because the file is easy to ignore when the pages themselves still render. By then, stale entries and missing URLs have usually been accumulating for a while.
AEOprobe gives you one place to check whether the sitemap is still aligned with the live crawl of the site.
Run the free audit now if you want to see whether your sitemap is reinforcing discovery or quietly creating crawl waste.
Questions
What is an XML sitemap audit?
An XML sitemap audit checks whether the sitemap is fetchable, valid, scoped to the right URLs, and useful as a discovery and freshness signal rather than just present on the site.
What are the most common sitemap problems?
Common problems include invalid or duplicate URLs, missing or invalid lastmod values, out-of-scope entries, crawled pages missing from the sitemap, and URLs that return 404 even though the sitemap still lists them.
Does submitting a sitemap guarantee indexing?
No. Google describes submitting a sitemap as merely a hint. A sitemap helps crawlers find the URLs you want in search results. It does not guarantee they are crawled or indexed.
What does AEOprobe report for sitemap quality?
AEOprobe reports whether the sitemap was fetched successfully and whether the XML was valid. It also reports how many URLs were listed and where duplicate, invalid, out-of-scope, missing, or broken entries and missing or invalid lastmod values were found.
Related posts
AI Search Audit: A Technical SEO Audit for AI Readiness
An AI search audit is a technical SEO audit that checks whether AI and search crawlers can reach and read your site.
Read moreHreflang Audit Checklist: How to Fix International SEO Signals That Break Discovery
Check hreflang self-references, return links, invalid codes, relative URLs and x-default. See what AEOprobe reports.
Read moreSchema Markup Audit: Each Structured-Data Check and the Block That Fixes It
What our audit checks in JSON-LD, in the words its report uses, with the example block that fixes each check.
Read moreProgrammatic SEO Audit: How to Catch Template Risk, Thin Pages, and Index Bloat
Group templated URLs into page families. Catch a weak template before it becomes a crawl and quality problem.
Read moreCheck what AI crawlers can actually reach
Run a free audit to find public crawler, indexing, markup, and page-structure issues.
Run free audit