Google explains why HTML sitemaps still have a purpose, even though the XML standard has largely replaced it.

Google’s John Mueller and Martin Splitt recently discussed how HTML sitemaps may still be useful for SEO even though the XML standard has largely replaced it.

Before the introduction of XML sitemaps, SEOs and site owners typically added internal and external links to new web pages to ensure that they got crawled and started ranking.

They also added a link from the footer of the home page (or every page) to an HTML sitemap where a gigantic list of links to inner pages could be crawled and aid in getting those pages indexed. Most SEOs populated the HTML sitemap with every page on their site that was considered useful to rank. Other pages, like the About Us pages, were left off of the HTML sitemap.

It’s possible that Google was already using RSS feeds well before the introduction of XML sitemaps. For example, a 2004 university research paper documented that RSS feeds could be used by web crawlers to identify new and recently modified web pages and reduce crawling bandwidth by up to 40%.

But because RSS feeds weren’t universally implemented, they were likely impractical to use for crawl discovery. That’s probably why GYM (Google, Yahoo, and Microsoft) joined forces to agree on a single XML sitemap standard that is used to this day.

During the course of their recent Search Off The Record episode, Martin Splitt asked if SEOs and site owners were locked into using XML sitemaps OR if it was okay to use HTML sitemaps.

Source link