Web Unlocker API: Render Any Page to HTML, Markdown, or PNG

TL;DR:

• The Web Unlocker turns any URL into clean data with one POST. Send a URL to unlocker.webunlocker; get back the page as HTML, plaintext, Markdown, a screenshot, or extracted content — no browser to manage.
• JavaScript rendering is a flag, not a separate product. Set jsRender.enabled: true and the page is rendered in a real browser before the response is built, so client-side content is already there.
• You pick the shape of the response. response.type is one of html, plaintext, markdown, png, jpeg, network, or content — request Markdown for LLMs, PNG for a screenshot, content for structured extraction.
• Proxy country is a field. Set proxy.country to route the request through residential egress in that region; a mismatched region is a common reason a page renders differently.
• Two timeouts govern every call. A 30-second page-load ceiling and a 180-second global execution ceiling — the page-load limit takes priority.
• Free to start. New Scrapeless accounts include free Universal Scraping API usage — sign up at app.scrapeless.com.

Introduction: one endpoint, any page, the shape you asked for

Most scraping code spends its effort on everything around the data: launching a browser, waiting for JavaScript, handling the block, then parsing HTML into something usable. The Scrapeless Universal Scraping API collapses that into a single HTTP request. You POST a URL to the Web Unlocker actor, and the response is the page — already rendered, already in the format you asked for.

This guide walks through the unlocker.webunlocker actor end to end: the request shape, a first curl, the response envelope, a Python integration, the seven response types JavaScript rendering can return, and how to keep requests clean. Every request and response below was captured against the live API.

What you can do with it

• Fetch a page as raw HTML — a plain GET through clean egress, for when you'll parse the markup yourself.
• Render JavaScript-heavy pages — set jsRender.enabled and read content that only exists after the client runs.
• Get Markdown for an LLM — request type: markdown and feed the result straight into a RAG pipeline or prompt.
• Capture a screenshot — request type: png or jpeg and get the rendered viewport as an image.
• Extract structured content — request type: content to pull headings, links, tables, emails, images, and metadata out of the page.
• Watch network responses — request type: network to capture the XHR/fetch responses a page makes, filtered by URL, status, and method.
• Drive the page first — run instructions (wait for a selector, click, fill, press keys) before the response is built.

Why Scrapeless Universal Scraping API

The Universal Scraping API is the managed web-unlocking surface: you send a URL, it handles rendering, egress, and anti-detection, and returns clean data. For this workflow specifically, it brings:

• Cloud-side JavaScript rendering — a real browser runs the page, so single-page apps and lazy-loaded content resolve before the response is built.
• Residential proxies in 195+ countries — route through proxy.country so the exit IP reputation reads clean and geo-routed pages serve correctly.
• Automatic challenge handling — reCAPTCHA v2, Cloudflare Turnstile, and the Cloudflare interstitial are handled inside the actor.
• Seven response formats — HTML, plaintext, Markdown, PNG, JPEG, network capture, and structured content from the same endpoint.
• A single HTTP contract — no browser lifecycle, no driver versions; the response is the data.

Get your API key on the free plan at app.scrapeless.com.

Prerequisites

• A Scrapeless account and API key — sign up at app.scrapeless.com
• curl for the first request, and Python 3.10+ (or Node.js 18+) for the integration
• Basic familiarity with HTTP and JSON

How the Web Unlocker works

Every call is a POST to one endpoint with a JSON body of {actor, input, proxy}.

Request parameters

Auth is the x-api-token header. The response envelope is always { "code": 200, "data": ... }.

Quick capture with curl

Fetch a page as HTML through residential egress:

curl -X POST https://api.scrapeless.com/api/v2/unlocker/request \
  -H "x-api-token: ${SCRAPELESS_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "actor": "unlocker.webunlocker",
    "input": { "url": "https://www.example.com", "method": "GET", "redirect": false },
    "proxy": { "country": "ANY" }
  }'

Response envelope

// illustrative sample — envelope shape is exact; the live GET above returned code 200 with 559 bytes of example.com HTML
{
  "code": 200,
  "data": "<!doctype html><html>…</html>"
}

A code of 200 means the request succeeded; data carries the payload — HTML text here, Markdown or a base64 image for other response types.

Integrating the API in Python

The same call from Python, reading the key from the environment:

import os
import requests

API_KEY = os.environ["SCRAPELESS_API_KEY"]

resp = requests.post(
    "https://api.scrapeless.com/api/v2/unlocker/request",
    headers={"x-api-token": API_KEY, "Content-Type": "application/json"},
    json={
        "actor": "unlocker.webunlocker",
        "input": {"url": "https://www.example.com", "method": "GET", "redirect": False},
        "proxy": {"country": "ANY"},
    },
    timeout=70,
)

data = resp.json()
if data.get("code") == 200:
    html = data["data"]
    print(len(html), "bytes of HTML")

Get your API key on the free plan: app.scrapeless.com

Rendering JavaScript: the seven response types

To render the page in a real browser first, add jsRender. The response.type decides what comes back. Request Markdown for a page — ideal for feeding an LLM:

payload = {
    "actor": "unlocker.webunlocker",
    "proxy": {"country": "ANY"},
    "input": {
        "url": "https://www.example.com",
        "jsRender": {
            "enabled": True,
            "response": {"type": "markdown"},
        },
    },
}
resp = requests.post(
    "https://api.scrapeless.com/api/v2/unlocker/request",
    json=payload,
    headers={"x-api-token": API_KEY, "Content-Type": "application/json"},
    timeout=70,
)
print(resp.json()["data"])
# "# Example Domain\n\nThis domain is for use in documentation examples..."

The type field selects the format:

For a screenshot, request png and decode the base64 data to bytes:

import base64

payload["input"]["jsRender"]["response"] = {"type": "png"}
resp = requests.post(
    "https://api.scrapeless.com/api/v2/unlocker/request",
    json=payload,
    headers={"x-api-token": API_KEY, "Content-Type": "application/json"},
    timeout=70,
)
with open("page.png", "wb") as f:
    f.write(base64.b64decode(resp.json()["data"]))

Driving the page before capture

When content appears only after interaction, pass instructions — each is a verb the renderer runs in order before building the response:

{
  "actor": "unlocker.webunlocker",
  "input": {
    "url": "https://example.com",
    "jsRender": {
      "enabled": true,
      "instructions": [
        { "waitFor": [".dynamic-content", 30000] },
        { "click": ["#load-more", 1000] },
        { "fill": ["#search-input", "search term"] },
        { "keyboard": ["press", "Enter"] },
        { "evaluate": "window.scrollTo(0, document.body.scrollHeight)" }
      ]
    }
  }
}

You can also cut bandwidth by blocking resource types you don't need with jsRender.block.resources (for example Image, Font, Media, Stylesheet), which the fetch layer skips per the resource categories defined in the Fetch API.

How to avoid common problems

• A field that isn't on the page is null, not an error. Treat every extracted field as optional and guard for its absence rather than assuming it's present.
• Mind the two timeouts. A page-load ceiling of 30 seconds and a global execution ceiling of 180 seconds bound every call, and the page-load limit takes priority — keep waitFor values inside that budget. The HTTP semantics specification defines the status codes you'll see if a target itself errors.
• Pin the country to the content. If a page geo-routes, set proxy.country to the region that serves the version you want; ANY is fine when it doesn't.
• Choose the response type deliberately. Request markdown or content when you want data, not html you'll have to parse — the extraction happens server-side either way, and the automated-traffic patterns the unlocker handles are catalogued in the OWASP Automated Threats project.

Conclusion: the page, in the shape you need

The Web Unlocker reduces a scrape to one decision: which URL, and which response type. Rendering, egress, and anti-detection are handled inside the actor, so a JavaScript-heavy page becomes clean Markdown or a screenshot in a single request. Pair it with the Scraping Browser when you need a full interactive session, and read up on residential versus datacenter egress since proxy reputation decides most render outcomes. The Universal Scraping API docs cover every field.

Ready to Build Your AI-Powered Data Pipeline?

Join our community to claim a free plan and connect with developers building extraction pipelines: Discord · Telegram.

Sign up at app.scrapeless.com for free Universal Scraping API usage, and see pricing for scale.

FAQ

Q: What is the difference between the Web Unlocker and the Scraping Browser?
The Web Unlocker is a single request/response endpoint — send a URL, get the page back in one call. The Scraping Browser is a full interactive cloud browser you drive with Puppeteer or Playwright. Use the unlocker for fetch-and-parse; use the browser for multi-step sessions.

Q: Do I need to enable JavaScript rendering?
Only when the content you need is client-rendered. A plain GET returns the server HTML; adding jsRender.enabled: true runs the page in a real browser first, which is what you want for single-page apps and lazy-loaded content.

Q: Which response type should I use for an LLM pipeline?
markdown — it returns the page as clean Markdown with markup stripped, which is what most RAG and prompt pipelines want. Use content when you need discrete fields (headings, links, tables) instead of prose.

Q: How do I get a screenshot?
Set response.type to png or jpeg; the data field comes back as a base64 string you decode to image bytes.

Q: Do I need a proxy?
Egress is built in. Set proxy.country to route through residential IPs in a specific region, or ANY to let the service choose. Pinning a country matters when a page geo-routes or challenges datacenter IPs.

Q: What are the timeouts?
A fixed 30-second page-load ceiling and a 180-second global execution ceiling. The page-load limit takes priority and can end the call before the global limit, so keep any waitFor values within that budget.