Does it find the sitemap for me?

No, you supply the sitemap or sitemap index URL directly. It does not read robots.txt to discover sitemap locations, so check there first if you do not know the URL.

What happens with malformed or broken sitemaps?

If XML parsing fails, a regex fallback pulls anything between loc tags, so you still get URLs, but lastmod, priority and changefreq are not captured in fallback mode. Errors fetching an individual child sitemap are shown in the status line and the run continues; failed sitemaps are not listed in the final export, so sanity check the Sitemaps Processed count.

Where does the request delay actually apply?

Between sitemap file fetches only. The listed URLs are never requested, so a 50,000 URL sitemap is a single request; the delay only matters for indexes with many child sitemaps.

It decompresses when the URL ends in .gz or when the response body starts with the gzip magic bytes, so compressed sitemaps served without the .gz extension still work.

What if the server blocks the tool?

The default user agent is "Mozilla/5.0 (compatible; SitemapExtractor/1.0)", which some hosts and CDNs reject. Change the user agent field to a full browser string if you see 403s. Each request also times out after 30 seconds.

Sitemap URL Extractor: free Technical SEO tool

#about

Dual-strategy XML parsing: ElementTree with namespace handling, regex fallback (<loc>.*?</loc>).

Recursive queue-based processing of sitemap indexes.

Gzip decompression for .xml.gz files.

Configurable request delay (0-5 seconds slider).

Duplicate prevention via processed_sitemaps set tracking.

Streamlit app

Platform: Browser-based (no installation required)
Input: Sitemap URL (index or individual .xml/.xml.gz)Custom user agent stringRequest delay: 0-5 seconds
Output: URL list with sitemap metadata (CSV/Excel)
Category: Technical SEO

Use cases

SEO audits - get complete URL inventory
Site migrations - document all URLs before changes
Competitor analysis - understand site structure
Content inventory - catalog all pages
Indexation monitoring - compare sitemap vs indexed

#features

Recursive sitemap index processing via queue
Dual parsing: ElementTree + regex fallback
Gzip decompression (.xml.gz support)
Duplicate sitemap prevention
Configurable delay slider (0-5 seconds)

#how-to-use

Enter sitemap URL in the sidebar
Set custom user agent if needed
Configure request delay (recommended for large sites)
Review extracted URLs with metadata
Download as CSV, TXT, or Excel (timestamped filenames)

#faq

Does it find the sitemap for me?: No, you supply the sitemap or sitemap index URL directly. It does not read robots.txt to discover sitemap locations, so check there first if you do not know the URL.
What happens with malformed or broken sitemaps?: If XML parsing fails, a regex fallback pulls anything between loc tags, so you still get URLs, but lastmod, priority and changefreq are not captured in fallback mode. Errors fetching an individual child sitemap are shown in the status line and the run continues; failed sitemaps are not listed in the final export, so sanity check the Sitemaps Processed count.
Where does the request delay actually apply?: Between sitemap file fetches only. The listed URLs are never requested, so a 50,000 URL sitemap is a single request; the delay only matters for indexes with many child sitemaps.
How is gzip handled?: It decompresses when the URL ends in .gz or when the response body starts with the gzip magic bytes, so compressed sitemaps served without the .gz extension still work.
What if the server blocks the tool?: The default user agent is "Mozilla/5.0 (compatible; SitemapExtractor/1.0)", which some hosts and CDNs reject. Change the user agent field to a full browser string if you see 403s. Each request also times out after 30 seconds.

/contact

Want me to run this for you? I run this tool as a managed service, or build something custom around your data. You get the insights without touching the code.

Send me a message →