Serpwise AI

Expose a Model Context Protocol (MCP) endpoint on your domain so AI agents like Claude, ChatGPT, and Perplexity can query your site content and products via structured tools.

MCP Server exposes a Model Context Protocol endpoint directly on your domain at /mcp. AI agents connect to this endpoint and use structured tools to search your content, browse your product catalog, and retrieve page details — all without scraping HTML.

When enabled, the edge proxy intercepts /mcp requests, authenticates the caller, and processes them securely. Your origin server never sees these requests.

MCP Server is a domain-level feature. When enabled, all five tools are available. Content must be crawled and indexed before search_content returns results — run a crawl first if your domain is new.

How It Works

AI Agent (Claude Desktop, ChatGPT, etc.)
  │
  ▼
customer-site.com/mcp  ──►  SerpWise Edge Proxy  ──►  Semantic Search & Content Engine
                              │
                              ▼
                         /.well-known/mcp.json  (auto-discovery)
                         /llms.txt              (AI agent discovery)

An AI agent discovers the MCP endpoint via /.well-known/mcp.json or /llms.txt
The agent sends an MCP initialization request to customer-site.com/mcp
The edge proxy authenticates the request via API key
The proxy processes the tool call (semantic search, page retrieval, etc.)
Results are returned to the agent as structured JSON

Available Tools

Tool	Description	Parameters
`search_content`	Semantic search across all crawled pages	`query` (required), `domain_id`, `limit` (1-50)
`get_page`	Get full markdown content of a specific page	`url_path` (required), `domain_id`
`search_products`	Semantic search across the product catalog	`query` (required), `domain_id`, `limit` (1-50)
`get_product_details`	Full enriched product data (30+ AI attributes)	`product_id` (required)
`list_categories`	Browse all product categories	`domain_id`

All queries are automatically scoped to your organization and domain — no cross-account data access is possible.

Enabling MCP Server

Navigate to your domain in the dashboard
Click the AI tab
Select the MCP Server sub-tab
Toggle Enable MCP Server
Click Save MCP Settings

Changes take effect immediately. The /mcp endpoint, /.well-known/mcp.json, and /llms.txt become active as soon as you save.

Discovery Endpoints

When MCP is enabled, two discovery endpoints are automatically served:

`/.well-known/mcp.json`

JSON manifest describing the MCP server, its endpoint URL, authentication requirements, and available tools. AI agents use this for automatic discovery.

`/llms.txt`

Plain text file following the emerging llms.txt standard. Describes the site and points AI agents to the MCP endpoint for structured access.

Authentication

MCP requests require a Bearer token — an API key with the products:read scope.

Go to Organization Settings → API Keys
Create a new key with the products:read scope
Include it in MCP requests as: Authorization: Bearer sw_prod_...

API keys can optionally be restricted to specific domains.

Connecting from Claude Desktop

Add this to your Claude Desktop MCP configuration (claude_desktop_config.json):

{
  "mcpServers": {
    "my-site": {
      "url": "https://example.com/mcp",
      "headers": {
        "Authorization": "Bearer sw_prod_your_api_key_here"
      }
    }
  }
}

Restart Claude Desktop. The five tools will appear in the tool picker.

Connecting via curl

Initialize a session:

curl -X POST https://example.com/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer sw_prod_your_api_key_here" \
  -d '{
    "jsonrpc": "2.0",
    "method": "initialize",
    "params": {
      "protocolVersion": "2025-03-26",
      "capabilities": {},
      "clientInfo": { "name": "my-agent", "version": "1.0" }
    },
    "id": 1
  }'

Call a tool:

curl -X POST https://example.com/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: Bearer sw_prod_your_api_key_here" \
  -d '{
    "jsonrpc": "2.0",
    "method": "tools/call",
    "params": {
      "name": "search_content",
      "arguments": { "query": "return policy", "limit": 5 }
    },
    "id": 2
  }'

Cloudflare Configuration

If your domain uses Cloudflare, AI agents will be blocked by Cloudflare's bot challenge (they can't execute JavaScript). You need to create a WAF rule to skip the challenge for MCP requests:

Go to Cloudflare Dashboard > Security > WAF > Custom Rules
Create a new rule:
- Name: Allow SerpWise MCP & AI Discovery
- If: URI Path starts with /mcp OR URI Path equals /.well-known/mcp.json OR URI Path equals /llms.txt
- Then: Skip (All remaining custom rules, or set to Allow)
Save and deploy

This ensures AI agents can reach the MCP endpoint and auto-discover it via the standard discovery files.

Rate Limits

MCP requests are subject to the API key's rate limits (configured per key):

Per minute: Default 60 requests
Per day: Default 10,000 requests

Rate limit headers (X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset) are included in responses.

MCP Server

On this page