Sitemaps
Auto-generate XML and HTML sitemaps from your crawled page index — with priority computation, include/exclude patterns, and gateway serving.
Sitemaps lets you auto-generate XML and HTML sitemaps from the pages the Edge Crawler has indexed. Configure once, and SerpWise generates, publishes, and serves your sitemaps directly from the gateway — no origin changes needed.
XML Sitemap
The XML sitemap follows the sitemaps.org protocol and is served by the gateway at a path you configure.
Configuration
Navigate to your domain dashboard, click the Sitemaps tab, and configure the XML sitemap:
| Setting | Description | Default |
|---|---|---|
| Enable XML Sitemap | Toggle sitemap generation on or off | Off |
| Path | URL path where the sitemap is served | /sitemap.xml |
| Include Patterns | Glob patterns for paths to include (e.g. /blog/*, /products/*) | All pages |
| Exclude Patterns | Glob patterns for paths to exclude (e.g. /admin/*, /staging/*) | None |
| Change Frequency | Default <changefreq> value for entries | weekly |
| Auto-regenerate | Automatically regenerate after each crawler run | On |
Include and Exclude Patterns
Use glob patterns to control which crawled pages appear in your sitemap:
*matches any characters within a path segment**matches across path segments- Exclude patterns take precedence over include patterns
For example, to include only blog and product pages:
- Include:
/blog/*,/products/* - Exclude:
/products/discontinued/*
Priority Computation
SerpWise automatically computes a <priority> value for each URL based on:
- Link depth — pages closer to the homepage receive higher priority
- Recency — recently updated pages are prioritized
- Homepage always gets priority
1.0
You don't need to set priorities manually — the system calculates them from your page index data.
Sitemap Index
For sites with more than 50,000 URLs, SerpWise automatically creates a sitemap index file that references multiple sitemap files. This follows the sitemap index protocol and ensures compliance with the 50,000 URL limit per sitemap file.
The index is served at the same configured path (e.g. /sitemap.xml), with individual sitemap files at numbered paths (e.g. /sitemap-1.xml, /sitemap-2.xml).
HTML Sitemap
In addition to the XML sitemap, you can generate an HTML sitemap — a human-readable page listing all indexed URLs, useful for visitors and as an additional crawl path for search engines.
| Setting | Description | Default |
|---|---|---|
| Enable HTML Sitemap | Toggle HTML sitemap generation | Off |
| Path | URL path where the HTML sitemap is served | /sitemap |
| Title | Page title for the HTML sitemap | Sitemap |
| Grouping | Group pages by directory structure or flat list | Directory |
The HTML sitemap respects the same include/exclude patterns as the XML sitemap.
Publishing
Manual Publish
Click Publish to generate and activate the sitemap immediately. The gateway starts serving it at the configured path right away.
Auto-regenerate
When enabled, the sitemap is automatically regenerated after each Edge Crawler run completes. This keeps your sitemap in sync with your page index without manual intervention.
Preview & Download
Before publishing, use Preview to see exactly what the generated sitemap will contain. You can also Download the XML file to inspect it locally or submit it to search engine webmaster tools manually.
Sitemaps require crawled pages to generate. Run the Edge Crawler first to build your page index, then configure and publish your sitemap.
Site-Wide SEO Dashboard
Aggregate SEO health across your entire domain — overall scores, grade distribution, top issues, content health, and CSV export.
Product Intelligence Engine
Transform product feeds into an AI-powered growth engine with 4-tier cascading extraction, deterministic staleness detection, event-driven feed sync, and a native MCP server for AI agents.