The AI Crawler Index MCP server,
connected in one click.
{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key. Connect the official AI Crawler Index MCP server to Claude, ChatGPT, or Cursor through gate — one gateway URL, no config files, security-checked.
Free to start · No credit card · No hosting needed
Add AI Crawler Index to your AI in one minute
Pick your client. Copy, paste, done — or let gate handle sign-ins and policies for you.
claude mcp add --transport http ai-crawler-index https://www.pathwren.workers.dev/c/mcp-registry-official/mcpRun in your terminal, then restart Claude Code.
AI Crawler Index · https://www.pathwren.workers.dev/c/mcp-registry-official/mcp
What your AI can do with AI Crawler Index
Real prompts you can type the moment it's connected — no setup, no docs to read.
“Use “classify_user_agent”: FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Identify the crawler behind a raw User-Agent header: operator, category, robot.”
“Use “lookup_crawler”: FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — The full index record for one crawler: operator, category, robots.txt token an.”
“Use “list_crawlers”: FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — The index, filtered or whole: each row says what the crawler is and what block.”
“Use “generate_robots_txt”: FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — A ready-to-paste robots.txt block for one stance, with the reasoning and the c.”
How secure is the AI Crawler Index MCP server?
Every server in the gate directory has to pass the same four checks before your AI can touch it. Here's how AI Crawler Index holds up.
Official endpoint
Operated by AI Crawler Index (workers.dev) at its documented MCP endpoint — not a third-party mirror or community re-host.
No sign-in needed
This server only exposes public data — no account and no credentials are involved when you connect.
Scanned before it goes live
gate checks every tool for prompt injection, hidden instructions, data-exfiltration hints, and over-broad permissions before the server is available through your gateway.
Watched for drift
If AI Crawler Index adds or changes tools later, gate re-scans and alerts you — before your AI acts on the change.
Connected in three steps
Connect your gate
Add one gateway URL to Claude, ChatGPT, or any MCP client. One-time setup, about two minutes.
Pick AI Crawler Index
Choose AI Crawler Index in the gate directory. No sign-in needed.
Use it anywhere
AI Crawler Index's tools are live in every AI client you've connected — with rules and logging built in.
Free to start · No credit card · No hosting needed
The actual tools AI Crawler Index exposes
Fetched live from the official endpoint and re-checked daily by gate — not marketing copy.
classify_user_agent — FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Identify the crawler behind a raw User-Agent header: operator, category, robot
lookup_crawler — FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — The full index record for one crawler: operator, category, robots.txt token an
list_crawlers — FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — The index, filtered or whole: each row says what the crawler is and what block
generate_robots_txt — FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — A ready-to-paste robots.txt block for one stance, with the reasoning and the c
is_verified_crawler_ip — FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Test an IPv4 or IPv6 address against every crawler prefix the operators themse
whats_changed — FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Freshness and change state: when the index was built, when each operator sourc
changes_since — FIRST CALL, needs nothing: {"name":"whoami","arguments":{}} — Only what changed since your last call: prefixes added or removed, upstreams f
whoami — {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} — POST that exact body to https://www.pathwren.work
+ 1 more tools once connected
Who runs the AI Crawler Index MCP server?
The AI Crawler Index MCP server at a glance
https://www.pathwren.workers.dev/c/mcp-registry-official/mcpQuestions, answered
Is there an official AI Crawler Index MCP server?
Yes — AI Crawler Index operates its own remote MCP server at www.pathwren.workers.dev. gate connects you to that official endpoint and adds security scanning, per-tool access rules, and an activity log on top.
How do I connect AI Crawler Index to Claude or ChatGPT?
Add gate's gateway URL to your AI client once, then pick AI Crawler Index in the directory. No account or sign-in is needed. From then on, AI Crawler Index's tools are available in every client you've connected to gate.
Do I need to host anything?
No. AI Crawler Index runs the server, and gate is fully managed. You just connect one URL — there's nothing to install, deploy, or maintain.
Is the AI Crawler Index MCP server safe to connect?
It's the vendor's official endpoint, and gate scans its tools for prompt injection and hidden instructions before it goes live, re-checks it whenever it changes, and lets you allow, block, or require approval for every single tool.
Is it free?
Yes — you can get started with gate for free and connect AI Crawler Index in minutes. No credit card required.
Connect more than just AI Crawler Index
Every server below works through the same gateway URL — connect once, add tools anytime.
Squarespace DomainsDesignView →Set up AI Crawler Index in: Claude · Claude Code · ChatGPT · Cursor · VS Code
Give your AI AI Crawler Index — safely.
One URL. All MCPs. Full control. Free to start.
Get started free