hreflang at Scale: Managing Large Multilingual Websites

Short answer: On large multilingual sites, hreflang must be generated from a single source of truth that records which pages are equivalents, not maintained by hand. For many languages and many URLs, XML sitemaps are usually the most practical delivery method, split into files within the 50,000-URL and 50 MB limits and listed in a sitemap index. Include only real translations in each cluster, regenerate on every change and monitor with regular crawls, because at scale, small template or data errors affect thousands of pages at once.

Why scale changes the problem

On a small site with a few languages, hreflang is a block of link tags that a plugin generates, and errors are rare and easy to spot. On a large site, such as a shop with tens of thousands of products in a dozen markets or a publisher with a large archive in many languages, the numbers change the nature of the task.

Consider a catalog of 20,000 products available in 12 store versions. If every product exists in every store, each page carries 13 hreflang entries (12 versions plus x-default), and the whole site contains over three million hreflang links. Every one of them must point to a live, canonical, indexable URL and be returned by its target. No team can check that by hand.

At this scale, hreflang is no longer markup; it is data. The quality of the result depends on the quality of the data that says which pages are equivalents, and on the process that turns that data into annotations.

Build one source of truth for equivalents

Every hreflang cluster is a statement: “these URLs are the same content for different audiences.” On large sites, that statement should come from one reliable place:

• Shops: a shared product ID or SKU across stores, with a record of which stores sell each product and the product’s URL in each store.
• Publishers and content sites: a translation group ID that links an article to its translations in the CMS.
• Category and landing pages: an explicit mapping table, because category structures often differ between markets.

Avoid deriving equivalents from URL patterns alone, such as assuming that /de/x/ and /fr/x/ are always the same page. Slugs are translated, categories are restructured and pages are removed, so pattern-based guesses break silently over time.

Choose the delivery method for scale

All three methods, HTML head, XML sitemap and HTTP header, are valid. At scale, the trade-offs become sharper:

• HTML head: simple and visible, but every page grows with every language. With many languages, the head becomes heavy, and every template must output the same correct cluster.
• XML sitemaps: keep pages light and can be generated in bulk from the data source. They are usually the most practical choice for large, many-language sites.
• HTTP headers: useful for documents such as PDFs, rarely for large numbers of HTML pages.

Whichever you choose, use one method consistently. Running head tags from one system and sitemap hreflang from another is a common cause of conflicting clusters on large sites.

Working within sitemap limits

Each sitemap file can contain at most 50,000 URLs and must be no larger than 50 MB uncompressed, as defined in the sitemaps protocol. hreflang entries make files much larger, because each URL entry includes one alternate link per version. In practice, sitemaps with hreflang often reach the size limit long before the URL limit.

• Split sitemaps into smaller files, for example by language and content type (products, categories, articles), and keep each file comfortably within the limits.
• Use a sitemap index that lists all sitemap files, and submit the index in Search Console for each property.
• Compress files with gzip to reduce transfer size; the uncompressed limit still applies.
• Include lastmod values that reflect real content changes, so search engines can prioritise recrawling what changed.
• Remember the cross-file rule: each URL listed as an alternate should also appear as its own url entry somewhere in the sitemaps, with the same set of alternates.

Partial clusters are normal

On large sites, not every page exists in every language or market. Products are sold only in some countries, articles are translated selectively, and categories differ. That is normal, and hreflang handles it well, as long as clusters reflect reality:

• A page with translations in three of twelve languages has a cluster of three (plus x-default if used).
• A page with no translations has no hreflang at all.
• Never fill gaps by pointing to category pages, home pages or the nearest similar product in another market. Those are not equivalents and contradict the return-link logic.

Keeping partial clusters accurate requires the data source to know, for each page, exactly which versions exist and are live.

Keep hreflang in sync with the site

Large sites change constantly. Products go out of stock or are discontinued, pages are renamed, markets are added. hreflang must follow:

1. Regenerate on change. When a URL changes or a page is removed, regenerate the affected sitemaps or templates the same day.
2. Use only live, indexable targets. Filter out URLs that redirect, return errors, are noindex or canonicalise elsewhere before writing them into hreflang.
3. Handle removals in all versions. When a product is removed from one store, every other store’s cluster for that product must drop the link to it.
4. Version the generation logic. Changes to the generator can affect millions of links; test them on a sample before deploying.

Common failures at scale

Ownership across teams and markets

On large international sites, hreflang touches several teams at once: developers maintain the generator, content teams create and remove pages, market teams decide which products are sold where, and SEO specialists check the results. When nobody owns the whole chain, errors fall between teams. The generator works as designed, but the data it receives is wrong, or the data is right but a template change broke the output.

A few organisational habits prevent most of this:

• Name one owner for hreflang as a system, usually in the technical SEO or platform team, who approves changes to the generator and to URL structures.
• Document the rules: which codes each market uses, what x-default points to, how partial clusters are handled and where the equivalence data comes from.
• Include hreflang in release checklists for any change to templates, URL routing, caching or sitemap generation.
• Give market teams a simple report showing hreflang errors for their market, so they notice problems in their own data, such as products launched without a proper mapping.

These steps cost little compared with the traffic at stake. On a large site, a single broken release can remove the correct version from search in several markets at once, and it may take weeks to notice without clear ownership.

Monitoring large sites

With millions of annotations, monitoring has to be systematic:

• Crawl regularly, including following hreflang links, and track error counts per language and page type over time. A sudden jump after a release points to a generator or template change.
• Sample by template. Check a few URLs per template and language manually after every release.
• Watch Search Console per market. Filter by country and folder to spot markets where the wrong version starts appearing.
• Compare sitemap URLs with crawled URLs. URLs in sitemaps that the crawl finds redirecting or missing indicate sync problems.

Site SEO AI Audit crawls up to the page limit of your plan, starting from the home page and the sitemap, and checks hreflang return links, broken language versions, x-default and lang attributes on every crawled page, weighting each issue by the share of pages it affects. Plans for larger sites include weekly audits, alerts and comparisons between audits, which suit post-release monitoring; see the plans.

Related reading

• hreflang in XML sitemaps vs HTML head vs HTTP headers
• XML sitemap best practices: what to include and leave out
• International ecommerce SEO: selling in several countries

The bottom line

At scale, hreflang is a data problem. Keep one source of truth for which pages are equivalents, generate annotations from it, usually in split XML sitemaps within the protocol limits, include only live, indexable, real translations, regenerate whenever the site changes and monitor with regular crawls. Small mistakes multiply on large sites, so the system matters more than any single tag.

FAQ

What is the best hreflang method for very large sites?

XML sitemaps are usually the most practical, because they keep pages light and can be generated in bulk from product or translation data. The HTML head also works if templates are reliable.

How many URLs can a sitemap with hreflang contain?

The protocol allows up to 50,000 URLs and 50 MB uncompressed per file. With hreflang, files often hit the size limit first, so split them into smaller files listed in a sitemap index.

Do all pages need to exist in every language?

No. Partial clusters are normal. Each page lists only the versions that actually exist and are indexable.

How often should hreflang sitemaps be regenerated?

Whenever URLs are added, changed or removed. On busy shops that often means daily or on every catalog update.

Can I derive hreflang from URL patterns?

It is risky. Translated slugs, removed pages and different category structures break pattern-based mapping. Use explicit IDs that link equivalent pages.