How to Scrape Bing News

TL;DR:

• Bing News results live in .news-card elements, and the clean data is in attributes, not text. Each card carries data-title, a url attribute, and data-author (the publisher) — read those instead of scraping the rendered text, which mixes in the timestamp.
• The page renders client-side, so you need a real browser. Bing builds the news grid with JavaScript after the initial load; a plain HTTP fetch returns the shell, not the cards.
• Pagination is an offset on the URL. Append &first=11, &first=21, … to walk deeper result pages — there's no "next" button to click, just the offset.
• Take the snippet from the title attribute. .snippet shows truncated text on screen but stores the full summary in its title attribute.
• Pin US residential egress for stable results. Bing localizes and rate-limits by IP; a consistent US residential session keeps the result set and markup predictable.
• Free to start. New Scrapeless accounts include free Scraping Browser runtime — sign up at app.scrapeless.com.

Introduction: turning Bing News into a structured feed

A Bing News search page is a live, query-driven feed — headlines, publishers, timestamps, and summaries for any topic you give it. That makes it useful for brand monitoring, market research, and feeding fresh articles to an AI agent. The catch is that none of it arrives as clean HTML you can grep.

The grid is built in the browser. Load bing.com/news/search with a plain HTTP client and you get the page scaffold, not the stories — the cards only appear after JavaScript runs. And once they do render, the on-screen text smears the publisher and the relative time together ("15h on MSN"), so reading innerText gives you data you then have to untangle.

This guide runs a Bing News scraper on Scrapeless Scraping Browser — an anti-detection cloud browser connected to Puppeteer — and reads the structured attributes Bing already attaches to each card. Every snippet below was run against a live search. Public results only.

What You Can Do With It

• Build a topic feed — pull every headline, publisher, link, and summary for a search term.
• Monitor a brand or product by re-running the same query on a schedule and diffing the results.
• Collect article URLs to hand off to a full-text crawler downstream.
• Walk deep result pages with the first= offset to gather far more than the first screen.
• Feed an AI agent a clean, structured news set instead of raw HTML.

Why Scrapeless Scraping Browser

Scrapeless Scraping Browser is a customizable, anti-detection cloud browser designed for web crawlers and AI agents. For Bing News specifically, it brings:

• Self-developed Chromium — renders the JavaScript that builds the news grid, so the cards actually exist in the DOM.
• Anti-detection fingerprinting — the session reads as a real browser, so the results page serves normally.
• Residential proxies in 195+ countries — pin US egress so Bing returns a consistent, localized result set.
• Session persistence — keep one session warm across paginated requests.
• A standard Puppeteer connection — Puppeteer.connect() returns a normal Browser; your extraction code is plain Puppeteer.

Get your API key on the free plan at app.scrapeless.com.

Prerequisites

• Node.js 18 or newer
• A Scrapeless account and API key — sign up at app.scrapeless.com
• Basic familiarity with Puppeteer

Install

npm install @scrapeless-ai/sdk puppeteer-core

export SCRAPELESS_API_KEY="your_api_token_here"

Step 1 — Connect and load the search

Connect to the cloud browser, pin US egress, and open the news search URL for your query. The query goes in the q parameter:

import { Puppeteer } from '@scrapeless-ai/sdk';

const browser = await Puppeteer.connect({
  apiKey: process.env.SCRAPELESS_API_KEY,
  sessionName: 'bing-news',
  proxyCountry: 'US',
  sessionTTL: 300,
});

const page = await browser.newPage();
const query = 'openai';
await page.goto(`https://www.bing.com/news/search?q=${encodeURIComponent(query)}`, {
  waitUntil: 'networkidle2',
  timeout: 60000,
});

console.log(await page.title());
// 'openai - Search News'

networkidle2 waits for the result requests to settle, so the cards are in the DOM before you read them.

Step 2 — Extract the cards from their attributes

Each result is a .news-card. Rather than parse the rendered text, read the structured attributes Bing attaches to the card — they give you the clean title, link, and publisher directly:

const articles = await page.evaluate(() => {
  const abs = (u) => { try { return new URL(u, location.href).href; } catch { return null; } };
  return [...document.querySelectorAll('.news-card')]
    .map((card) => ({
      title: card.getAttribute('data-title'),
      url: abs(card.getAttribute('url')),
      source: card.getAttribute('data-author'),
      snippet:
        card.querySelector('.snippet')?.getAttribute('title') ||
        card.querySelector('.snippet')?.innerText?.trim() ||
        null,
    }))
    .filter((a) => a.title && a.url);
});

console.log(articles.length, 'articles');
console.log(articles[0]);
// 12 articles
// {
//   title: "Japan's tech business SoftBank rolls out OpenAI 'patches' against cyberattacks",
//   url: 'https://apnews.com/article/openai-softbank-japan-technology-intelligence-cyberattacks-d8d3f9b2e5042ea949a7d5c53b782d96',
//   source: 'Associated Press News',
//   snippet: 'Japanese technology giant SoftBank Group Corp. is launching a service ...'
// }

data-author is the publisher name on its own ("Associated Press News"), without the "15h on MSN" timestamp the visible text carries. The .snippet element truncates on screen but keeps the full summary in its title attribute, so read that first.

Get your API key on the free plan: app.scrapeless.com

Step 3 — Page through deeper results

Bing News paginates with a first= offset on the URL — there's no next button, you just step the offset by ten. Reuse the same session and dedupe by title, since Bing reorders some cards across pages:

const seen = new Map();

for (const first of [1, 11, 21]) {
  await page.goto(
    `https://www.bing.com/news/search?q=${encodeURIComponent(query)}&first=${first}`,
    { waitUntil: 'networkidle2', timeout: 60000 }
  );

  const batch = await page.evaluate(() =>
    [...document.querySelectorAll('.news-card')]
      .map((card) => card.getAttribute('data-title'))
      .filter(Boolean)
  );

  for (const title of batch) seen.set(title, true);
  console.log(`first=${first}: ${batch.length} cards, ${seen.size} unique so far`);
}
// first=1:  10 cards, 10 unique so far
// first=11: 10 cards, 18 unique so far
// first=21: 10 cards, 26 unique so far

Each offset returns about ten cards; the running unique count climbs as you advance. Keep stepping until the unique total stops growing — that's the end of the result set for the query.

What You Get Back

Each article is a flat record — title, link, publisher, and summary:

[
  {
    "title": "SoftBank launches cybersecurity product based on OpenAI models",
    "url": "https://www.msn.com/en-in/lifestyle/style/softbank-launches-cybersecurity-product-based-on-openai-models/ar-AA25LC2i",
    "source": "The Economic Times on MSN",
    "snippet": "The \"Patching as a Service\" product will be rolled out in Japan through a joint venture ..."
  }
]
// Schema reflects exactly what the Step 2 eval emits. Field values are illustrative samples.

A few honest observations:

• The result count per page is not fixed. The first screen rendered about a dozen cards; offset pages returned ten each. Count what you get, don't assume a page size.
• Some cards repeat across offsets. Bing reorders results, so dedupe by title or URL as you paginate.
• Publisher format varies. data-author is usually the clean name, but syndicated stories read "… on MSN"; normalize if you key on publisher.
• Links point off-site. The url attribute resolves to the publisher's article, not a Bing redirect — good for handing straight to a full-text crawler.

Conclusion: a structured Bing News feed

A reliable Bing News scraper is render → read the card attributes → step the first= offset. Render in a real browser so the grid exists, pull data-title/url/data-author instead of untangling the visible text, and paginate by offset while deduping. Running on Scrapeless Scraping Browser supplies the Chromium rendering and residential egress that make the results load consistently. To turn the collected links into full article bodies, pair this with a site scraper built the same way; the Scraping Browser product page and docs cover the full SDK surface. Pin US egress, read the attributes, and walk the offset.

Ready to Build Your AI-Powered Data Pipeline?

Join our community to claim a free plan and connect with developers building news and search pipelines: Discord · Telegram.

Sign up at app.scrapeless.com for free Scraping Browser runtime and adapt the patterns above to the queries and regions your monitoring needs. See pricing for scale.

FAQ

Q: Is scraping Bing News legal?
Collecting publicly visible search results is generally permissible, but Bing's Terms of Service and local law still apply to how you store and use the data. Scrape only public results, respect the robots and ToS rules, and consult counsel for your use case.

Q: Do I need a proxy?
Yes — pin proxyCountry: 'US' (or your target region). Bing localizes and rate-limits by IP, so a consistent residential egress keeps the result set and markup stable.

Q: I get the page but no cards. What's wrong?
The cards are rendered by JavaScript. A plain HTTP fetch returns only the shell; load the page in the Scraping Browser and wait for networkidle2 so the grid is in the DOM before you read it.

Q: How do I get more than the first page of results?
Step the first= offset on the URL — &first=11, &first=21, and so on — reusing the same session, and dedupe by title since some cards reappear across pages.

Q: The markup changed and my selectors broke. Why?
Bing updates its layout periodically. Re-check the card attribute names (data-title, url, data-author) against a live page and tighten your selectors when they shift.

Q: How many parallel queries can I run?
Keep concurrency modest — three sessions per host is a safe ceiling for parallel runs against the same site.

Q: Can I run this without an AI agent?
Yes. It's plain Puppeteer over the Scrapeless session — no agent required.