Microsoft Copilot Scraper API: Capture Answers and Citations

TL;DR:

• A Copilot scraper API turns Microsoft Copilot's answer into structured JSON. One POST to the scraper.copilot actor returns the response text, the citations behind it, and the bare links it surfaced — as fields, not a screenshot.
• Three inputs run the whole thing. prompt carries the question, an optional country pins the run to residential egress in that market, and an optional mode selects Copilot's answer style.
• Citations arrive ready to chart. citations lists every cited source as { title, url } — the raw material for share-of-citation tracking without a parsing step.
• The envelope never changes. Every call returns { status, task_id, task_result }, the same shape as the other Scrapeless LLM actors, so a wrapper written for Copilot extends to ChatGPT, Grok, Gemini, and Perplexity unchanged.
• No browser to babysit. Rendering, session handling, and proxy rotation run server-side; you call one endpoint with an x-api-token header and read JSON back.
• Free to start. New Scrapeless accounts include free trial credits — sign up at app.scrapeless.com.

Introduction: Copilot answers where the buyer already works

Microsoft Copilot sits inside Windows, Edge, Microsoft 365, and Bing — so for a large share of business users, Copilot is the assistant they ask first. When that user asks for the best CRM, the best proxy provider, or the best help-desk tool, Copilot returns a short synthesized answer with a handful of cited sources. A brand is either named in that answer, or it is invisible to that buyer.

Tracking that answer by hand does not scale: the response is generated fresh each time, the cited sources rotate, and the wording shifts run to run. To monitor it as data, you need the answer and its citations as structured fields.

This guide walks through the scraper.copilot actor on the Scrapeless Scraper API — a single authenticated POST that returns Copilot's answer text, its citations, and its links as JSON, on the same envelope as the rest of the Scrapeless LLM-answer line.

What You Can Do With It

• Track brand visibility in Copilot. Run a fixed set of buying-intent prompts and check whether your brand appears in the answer text and the citation list.
• Measure share of citation. Group the citations URLs by domain to see which sources Copilot leans on for a topic, and where you rank against them.
• Compare Copilot to the other engines. Send the same prompt to scraper.copilot, scraper.chatgpt, scraper.gemini, and scraper.perplexity and diff the answers and sources side by side.
• Monitor answer drift. Capture the same prompt on a schedule and chart how the answer and its citations change over weeks.
• Feed downstream pipelines. The structured result_text and citations drop straight into a database, a dashboard, or an LLM-evaluation set.

Why the Scrapeless Copilot Scraper

Copilot is a JavaScript application behind Microsoft authentication and anti-automation defenses; a raw HTTP request returns nothing useful, and driving a real browser session yourself means rendering, sign-in, and proxy rotation to maintain. The Scrapeless Copilot Scraper — part of the Universal Scraping API — runs that surface server-side and hands back clean JSON. For Copilot specifically, it brings:

• Residential egress in 195+ countries, so a country value captures the answer a real user in that market would see.
• Cloud-side rendering and session handling — no browser to run, sign into, or keep alive.
• A stable response envelope shared with every other Scrapeless LLM actor, so one client covers the whole line.
• Citations as first-class fields — { title, url } per source, ready to group and count.

Get your API key on the free plan at app.scrapeless.com.

Prerequisites

• A Scrapeless account and an API key (the free plan includes trial credits) — app.scrapeless.com.
• The key exported as an environment variable so it never lands in source:

export SCRAPELESS_API_KEY="your_api_token_here"

• curl for a first capture, and Python 3 with requests for the worked client below.

How the Copilot Scraper works

You name the actor, hand it an input, and send your key in one header.

• Endpoint: POST https://api.scrapeless.com/api/v2/scraper/execute
• Actor: scraper.copilot
• Auth header: x-api-token: $SCRAPELESS_API_KEY

Request parameters

Quick capture with curl

curl -sS -X POST https://api.scrapeless.com/api/v2/scraper/execute \
  -H "Content-Type: application/json" \
  -H "x-api-token: ${SCRAPELESS_API_KEY}" \
  -d '{
    "actor": "scraper.copilot",
    "input": { "prompt": "What are the best proxy providers in 2026?", "country": "US", "mode": "smart" }
  }'

Response envelope

// illustrative sample — schema from a live scraper.copilot run; values abridged
{
  "status": "success",
  "task_id": "…",
  "task_result": {
    "prompt": "What are the best proxy providers in 2026?",
    "mode": "smart",
    "result_text": "The best provider depends on your use case… [1]",
    "citations": [
      { "title": "10 Best Proxy Providers for 2026: Tested & Ranked", "url": "https://…" }
    ],
    "links": [
      "https://…"
    ]
  }
}

Field by field:

Get your API key on the free plan: app.scrapeless.com

Integrating the API in Python

A complete client: send the prompt, check the envelope, and print the citation table.

import os
import requests

ENDPOINT = "https://api.scrapeless.com/api/v2/scraper/execute"


def ask_copilot(prompt: str, country: str = "US", mode: str = "smart") -> dict:
    resp = requests.post(
        ENDPOINT,
        headers={
            "Content-Type": "application/json",
            "x-api-token": os.environ["SCRAPELESS_API_KEY"],
        },
        json={"actor": "scraper.copilot", "input": {"prompt": prompt, "country": country, "mode": mode}},
        timeout=180,
    )
    resp.raise_for_status()
    return resp.json()


if __name__ == "__main__":
    data = ask_copilot("What are the best proxy providers in 2026?")
    result = data.get("task_result", {})
    citations = result.get("citations") or []
    print(f"status={data.get('status')} mode={result.get('mode')} citations={len(citations)}")
    for i, c in enumerate(citations, 1):
        print(f"  [{i}] {c.get('title', '')[:60]} → {c.get('url', '')[:60]}")

The answer body stays in result.get("result_text") as markdown; for share-of-citation work the loop above is usually the whole job — group the printed URLs by domain and count.

Companion actors for the rest of the AI-answer landscape

The same endpoint, header, and envelope cover the neighboring platforms — only the actor name and a platform-specific field or two change:

• scraper.chatgpt — same prompt/country input; returns result_text plus a content_references citation array and the search_result panel.
• scraper.grok — adds a required reasoning mode and returns separate web_search_results and x_search_results citation panels.
• scraper.gemini — same two-field input as ChatGPT; returns result_text plus a citations array.
• scraper.perplexity — takes a required country and a web_search flag; returns web_results, media_items, and related prompts.
• scraper.overview / scraper.aimode — Google's AI Overview block and AI Mode tab; the AI Overview guide covers that pair end to end.

Pricing for the line is usage-based with free trial credits on signup — current tiers are on the pricing page.

How to avoid common problems

• Empty citations on some prompts. Copilot does not cite sources for every answer — opinion-flavored or purely generative prompts can come back citation-free. For citation tracking, phrase prompts the way a researching buyer would ("best X for Y"), which reliably triggers web-grounded answers.
• Answers vary run to run. The same prompt can produce a different answer and citation set minutes apart — that volatility is the phenomenon you are measuring. Store every capture with its task_id and timestamp and treat the series, not any single run, as the signal.
• Treat every field as nullable. links is often empty and citation counts swing between runs. Read what is present rather than asserting a fixed shape.
• Pin the country deliberately. An unpinned run captures an answer; a pinned run captures the answer for a market you care about. Keep the country value in your stored records so series stay comparable.

Conclusion: Copilot answers as a one-line dependency

Copilot is a first-stop assistant for a large base of Windows, Edge, and Microsoft 365 users, and the answer it returns decides whether your brand is seen. The scraper.copilot actor turns that answer into a structured record — text, citations, links — on the same envelope as every other Scrapeless LLM actor, so monitoring Copilot is a one-line addition to a pipeline you may already run for ChatGPT or Gemini.

Ready to Build Your AI-Answer Data Pipeline?

Join our community to claim a free plan and connect with developers building AI-answer pipelines: Discord · Telegram.

Sign up at app.scrapeless.com for free trial credits, and point the scraper.copilot actor at the prompts, markets, and schedules your monitoring program needs.

FAQ

Q: Is scraping Microsoft Copilot answers legal?

A: The actor reads publicly available answer content, the same a visitor sees. As with any scraping, restrict use to public data, respect the platform's terms, avoid collecting personal data, and consult a lawyer if a use case is unclear.

Q: Do I need a Microsoft account or to handle sign-in?

A: No. Authentication, session handling, and rendering run server-side; you send a prompt and read JSON back.

Q: Why are the citations different each time I run the same prompt?

A: Copilot generates answers dynamically and re-selects sources per run, so the citation set shifts. That run-to-run variance is the signal a monitoring series is built to track — store each capture with its task_id and timestamp.

Q: Can I capture the answer for a specific country?

A: Yes. Pass a two-letter country code in the input to pin the run to residential egress in that market, so you capture the answer a local user would see.

Q: How is this different from the ChatGPT or Gemini scraper?

A: Only the actor name and a field or two. scraper.copilot returns its citations under citations; the envelope { status, task_id, task_result } is identical, so one client covers the whole LLM-answer line.