An XML sitemap is a file that lists the URLs of a site to facilitate their exploration by search engine bots. Locating this file on a third-party site (or on one’s own) allows for an assessment of the volume of indexable pages, the frequency of updates, and the technical structure of a domain. The methods to access it vary depending on the server configuration and the CMS used, and the sitemap URL is not always what one might guess.
Robots.txt: the first file to check to locate a sitemap
Most guides recommend directly testing /sitemap.xml at the root of the domain. This approach often works, but it overlooks a common case: the sitemap is not always hosted at this location. Some sites use custom paths, subdirectories, or file names generated automatically by their CMS.
Recommended read : How to Easily Download a Carrefour Invoice Online: Guide and Practical Tips
The robots.txt file, accessible at yourdomain.com/robots.txt, contains one or more lines starting with Sitemap: followed by the exact URL. This declaration is standardized and readable by all crawlers. It remains the most reliable source because it reflects what the site administrator has chosen to declare to the engines.
A single robots.txt can also point to multiple sitemaps, for example, a main sitemap and a sitemap dedicated to images or videos. Trying to guess the URL without consulting this file is akin to ignoring information that is already available. Starting there also allows those who wish to find the sitemap of Hi Business to check in a few seconds if the sitemap is publicly declared.
Read also : How to Easily Find Subscription Requests Sent on Instagram
Sitemap paths according to the CMS: WordPress, Shopify, Wix, and others
When robots.txt does not mention any sitemap, the site’s CMS provides a valuable clue. Each platform generates its file at a default location, and these locations differ significantly.
- WordPress without SEO plugin: since recent versions, WordPress generates a native sitemap accessible at
/wp-sitemap.xml. However, if Yoast SEO or Rank Math is installed, the path often becomes/sitemap_index.xml. - Shopify consistently places its sitemap at
/sitemap.xml, but the file is an index that points to sub-sitemaps (products, collections, pages, blog posts). - Wix also uses
/sitemap.xml, with a comparable index structure. Squarespace and Webflow follow the same convention. - Prestashop, in several common configurations, uses
/1_index_sitemap.xml, a path that few SEO professionals spontaneously test.
Identifying the CMS before testing URLs avoids blindly multiplying attempts. Tools like BuiltWith or Wappalyzer can detect a site’s technology in a matter of seconds.
Google Search Console and the gap between declared sitemap and considered sitemap
For a site that you own or have administrative access to, Google Search Console displays the sitemap actually submitted, its processing status, and the date of the last bot visit. This information goes beyond the mere existence of the file.
A rarely discussed point: the sitemap submitted in Search Console may differ from that declared in robots.txt. An old sitemap may remain referenced in robots.txt while a more recent version has been manually submitted via the console. The opposite can also occur, a sitemap declared in robots.txt but never submitted in Search Console, which does not prevent it from being crawled by Googlebot, but reduces visibility on its actual processing.
Bing Webmaster Tools offers a comparable function. For a complete SEO audit, cross-referencing the two sources helps identify inconsistencies between what the site announces and what the engines actually process.
What the sitemap status reveals
Search Console indicates the number of URLs detected in the sitemap and the number of URLs actually indexed. A significant gap between these two figures signals a technical problem: pages with 404 errors, looping redirects, contradictory noindex tags with presence in the sitemap, or content deemed insufficient by Google.
This diagnosis is impossible to perform from the outside, by consulting only the raw XML file. The sitemap alone guarantees nothing about actual indexing.
Automated sitemap discovery tools: usefulness and limitations
Several online tools automatically scan robots.txt, test conventional paths, and display metadata such as the number of URLs or the date of last modification. Solutions like Sitemap Finder (offered by ToolScraper or Pixellize) allow obtaining this information without manual intervention.
Their main advantage is the time saved during an audit covering multiple domains. However, these tools do not detect sitemaps protected by authentication or those served only via Search Console without public declaration. Field feedback varies on the reliability of some of them against atypical server configurations (URL rewriting, intermediate CDN, application firewall).
For occasional use on a single site, manually checking robots.txt followed by a URL test remains just as quick and more reliable. Automation makes sense when analyzing around ten domains simultaneously.
Sitemap and SEO: what the XML file changes concretely
Having a properly structured sitemap facilitates the exploration of pages by bots, but a sitemap does not force the indexing of a page. Google treats the file as a suggestion, not as a directive. A page excluded by a noindex tag or blocked by robots.txt will not be indexed even if it appears in the sitemap.
The real contribution of the sitemap lies in three areas:
- Indicating to engines deep or orphan pages that natural crawling (through internal links) would not easily discover.
- Communicating the last modification date via the
lastmodtag, which can speed up the recrawl of updated content. - Structuring URLs by content type (articles, products, images, videos) through separate sitemaps, which facilitates diagnosis in Search Console.
A sitemap containing error URLs or low-quality pages can, however, dilute the crawl budget. It is better to have a streamlined sitemap than a comprehensive sitemap including pages without value for SEO.
Locating the sitemap of a third-party site remains an analysis step, not an end in itself. What matters next is to confront the content of the file with the actual indexing data and the internal link structure of the domain.