Agent Discovery

The docs site serves AI agents a Markdown version of every page, a Markdown sitemap and a set of /.well-known discovery documents, so agents can find and read content without parsing HTML. This page lists every endpoint, the negotiation rules and what the layer adds.
View source

The agent discovery layer comes from nuxt-agent-discovery and is already configured by the layer. It lets AI agents read your docs as Markdown instead of HTML and find the MCP Server, llms.txt and Agent Skills through discovery documents at standard locations.

Endpoints

EndpointContent
/raw/**.mdThe Markdown version of each page
/sitemap.mdA Markdown index of every page
/.well-known/api-catalogAn RFC 9727 linkset of the service documents the site publishes
/.well-known/mcp/server-card.jsonThe MCP server card, served while MCP is enabled
/.well-known/skills/index.jsonThe Agent Skills index, served when the site has skills
/robots.txtAllow groups and a Content-Signal line for known AI agents

The homepage response carries a Link header pointing at these documents, so an agent requesting / gets the full list.

Markdown pages

Content negotiation

A page URL answers Markdown when, in this order:

  1. The URL ends in .md, such as /en/docs/getting-started.md
  2. Accept gives text/markdown a weight no lower than text/html
  3. Accept refuses text/html outright (text/html;q=0) while a wildcard allows Markdown
  4. The User-Agent is a known AI agent (such as ClaudeBot or GPTBot)

A bare */* does not count as asking for Markdown, so browsers always get HTML. /_, /api/, /mcp, /.well-known/ and static assets never negotiate. Responses carry Vary: Accept, User-Agent so CDNs cache the two representations separately.

Terminal
# Explicit .md URL
curl https://docs.example.com/en/docs/getting-started.md

# Negotiate with Accept
curl -H 'Accept: text/markdown' https://docs.example.com/en/docs/getting-started

# Request as an agent
curl -A 'ClaudeBot/1.0' https://docs.example.com/en/docs/getting-started

On Vercel, prerendered pages negotiate at the edge; on other hosts and in pnpm dev, a Nitro middleware does the same.

Document format

The response opens with frontmatter followed by the page body, with every same-origin link made absolute:

---
title: "Introduction"
description: "..."
canonical_url: "https://docs.example.com/en/docs/getting-started"
---

MDC components are converted to plain Markdown by the layer first (see What the layer adds), so code samples, tables and component APIs read as ordinary Markdown. A path naming a section rather than a page redirects 302 to the section's first page.

Homepage

/raw/index.md is generated by the layer instead of dumping the landing page's component markup. It contains the site title and summary, a ## Resources for Agents list built from the same registry as the Link header, and links to the site and its GitHub repository:

Terminal
curl https://docs.example.com/raw/index.md

Markdown sitemap

/sitemap.md lists every page by section, linking to each page's Markdown. The layer expands the first-level directories under /docs, one group each. Group titles are derived from directory names and can be overridden with labels:

nuxt.config.ts
export default defineNuxtConfig({
  agentDiscovery: {
    sitemap: {
      markdown: {
        expand: ['/docs'],
        labels: { 'getting-started': 'Getting Started', 'ai': 'AI' }
      }
    }
  }
})

The XML sitemap is still generated by @nuxtjs/sitemap, with /raw/** left out.

Discovery documents

API catalog

/.well-known/api-catalog answers application/linkset+json, grouping the MCP endpoint, llms.txt, skills and the docs home into a linkset:

Terminal
curl https://docs.example.com/.well-known/api-catalog

MCP server card

While MCP is enabled, /.well-known/mcp/server-card.json describes the server URL, transport, whether authentication is required, and the tools, resources and prompts read live from @nuxtjs/mcp-toolkit:

server-card.json
{
  "serverInfo": {
    "name": "my-docs",
    "title": "my-docs MCP Server",
    "documentation": "https://docs.example.com/docs/ai/mcp",
    "version": "1.0.0",
    "repository": "https://github.com/owner/repo"
  },
  "endpoints": [{ "type": "streamable-http", "url": "https://docs.example.com/mcp" }],
  "authentication": { "required": false }
}

The endpoint comes from aiChat.mcpPath (/mcp by default) and the documentation URL from mcp.browserRedirect. The name, description, version and repository in serverInfo are filled in per request:

FieldSource
namesite.name, then appConfig.seo.title
descriptionappConfig.seo.description
versionruntimeConfig.public.version
repositoryappConfig.github.url

robots.txt

The robots.txt generated by @nuxtjs/robots gets an Allow group for each known AI agent and a Content-Signal: search=yes, ai-train=yes, ai-input=yes line. The agent list is the same one content negotiation uses.

Error pages

When an agent or a command-line client requests a missing page, the 404 body is Markdown with recovery links such as the homepage and the sitemap; browsers and fetch() still get the HTML or JSON error they expect.

Terminal
curl https://docs.example.com/docs/not-found

What the layer adds

The layer only implements what the module cannot know:

  • MDC conversion: server/plugins/agent-discovery.ts hooks agent-discovery:document to turn MDC components into plain Markdown; component APIs such as ComponentProps become commented TypeScript interfaces
  • Server card metadata: the same plugin hooks agent-discovery:mcp-server-card to fill in serverInfo
  • Homepage: server/routes/raw/index.md.get.ts takes over /raw/index.md
  • Configuration: the agentDiscovery block in nuxt.config.ts sets the sitemap grouping and the skills directory, and leaves siteUrl and siteName empty so your site.url and site.name decide them

To change anything, override agentDiscovery in your project's nuxt.config.ts or register the same hooks in server/plugins/. See the nuxt-agent-discovery docs for every option.

site.url determines every absolute URL in the discovery documents. Make sure it is the production URL before a production build, see Deployment.
Copyright © 2024 - 2026