SeoSpidie
See your site the way search engines do
Type your website's URL — SeoSpidie first checks for an existing sitemap.xml (via robots.txt or the default location), which is faster and more complete than crawling. If none exists, it falls back to crawling every internal link it finds until there are no new pages left — or paste your own list manually. Larger sites can take a while since every page goes through the same fetch process as the rest of SeoSpidie.
Couldn't fetch that page automatically.
Paste its HTML source instead — right-click the page → View Page Source → copy all — and links will be extracted from that.
An XML sitemap is a file that lists the pages on your website that you want search engines to find. Each entry holds a page address and, optionally, the date it was last changed. Search engines read the sitemap alongside the links they discover while crawling, which helps them find new pages sooner and notice when existing pages are updated.
Sitemaps matter most for new websites with few links pointing to them, large sites with thousands of pages, and sites where some pages are only reachable through search or filters. They are also a diagnostic tool: when you submit a sitemap in Google Search Console or Bing Webmaster Tools, the reports show how many of the listed pages are actually indexed.
A good sitemap only contains canonical pages that return a normal 200 status — no redirects, no 404 pages, no noindexed pages and no duplicate parameter URLs. A single sitemap file can hold up to 50,000 URLs; larger sites split their URLs across several files and list them in a sitemap index.
An XML sitemap lists the URLs you want search engines to know about. It speeds up discovery of new pages and is especially important for large sites, new sites with few links, and pages that are hard to reach through navigation.
The page count tells you how many URLs were found. If the number is much lower than the pages you know exist, some pages are not linked from anywhere on your site — they are orphan pages, and they deserve internal links. If it is much higher, crawling probably found parameter or duplicate URLs that should not be in a sitemap.
After uploading, submit the sitemap address in Search Console and Bing Webmaster Tools, add a Sitemap line to robots.txt, and watch the indexing reports over the following weeks.
A management consultant in Toronto had a hand-coded website with 40 pages and no sitemap. Search Console showed only 22 pages indexed, and two new service pages had never appeared in Google.
The generator found no existing sitemap, so it crawled the site's internal links and collected 36 URLs. Comparing that list with the site's files revealed four pages that no other page linked to — including the two new service pages. The consultant linked them from the services menu, regenerated the sitemap, uploaded it and submitted it in Search Console and Bing Webmaster Tools. Within a few weeks all 40 pages were indexed.
Google says it ignores them. Accurate lastmod dates are more useful.
No. It helps discovery; each page still needs to be worth indexing.
SeoSpidie's server reads your public sitemap or crawls your public pages to list the URLs. Pasted lists are processed in your browser.
Upload it to the root of your site, for example yourdomain.com/sitemap.xml, add a Sitemap line to robots.txt, and submit the address in Google Search Console and Bing Webmaster Tools.
Up to 50,000 URLs or 50 MB uncompressed. Larger sites split their URLs into several sitemaps and list them in a sitemap index file.
Only pages you want in search results: canonical pages that load normally. Leave out redirects, error pages, noindexed pages and duplicate parameter URLs.
To read other websites, SeoSpidie's server fetches the public URLs you enter and sends the result back to your browser. We don't keep your reports or require an account. Text you paste stays in your browser. Fair-use limits apply to protect the service.
Yes. The XML Sitemap Generator is free to use with no sign-up. SeoSpidie is supported by advertising.
