Best Web Scraping APIs in 2026: 12 Compared by Job and Cost

Match the API to the unit of work. A cheap page API wins on static HTML; a render flag or hosted browser on JavaScript; a SERP or ecommerce endpoint when you want parsed records; a markdown-focused API for LLM pipelines. Shortlist two or three providers, then compare cost per usable page on your targets.

A web scraping API is a hosted endpoint that takes a URL (or a query) and returns the page or the data, while the provider runs proxies, retries, browsers, and block handling. You pay per request or credit; parsing and storage stay with you unless the API returns structured fields.

A typical request path:

1. Your client sends a URL, auth, and any geo or render flags.
2. The provider picks a proxy type and location for the target.
3. If JavaScript must run, a headless browser loads the page.
4. CAPTCHAs, challenges, and empty bodies trigger retries or alternate routes.
5. You get HTML, JSON, selected fields, or a structured error.

HTTP status and usable content are separate checks. The HTTP semantics standard (RFC 9110) describes what status codes mean for the exchange; it does not tell you whether the body contains the product price you need. A block page can still return HTTP 200.

If you would rather run a library or a point-and-click tool, our roundup of the best web scraping tools covers frameworks and no-code options.

Start from the job and the output your pipeline needs, then check how each provider bills that job (multipliers, failed requests), then test on a sample of your real URLs.

You need a cheap base request, enough concurrency, and geotargeting that matches your sources. Rendering is wasted cost here. Some APIs render by default: ScrapingBee documents JavaScript rendering as the default at 5 credits per request. Turn rendering off when the HTML is already in the first response.

Use a render flag when content appears after the initial load (render_js=1 on WebScrapingAPI's Scraper API, off by default). For waits, clicks, or screenshots, a hosted browser you control through a REST API saves you running your own browser fleet. Hosted browsers are usually driven through the Chrome DevTools Protocol.

Sites defend automated traffic for good reasons. OWASP's catalog of automated threats lists scraping as OAT-011. A scraping API changes the economics by managing IP reputation, browser fingerprints, and retries for you. Define success as validated content: WebScrapingAPI documents that a CAPTCHA page returned as HTTP 200 may be classified as a 422 failed scrape rather than valid content. Our playbook on web scraping without getting blocked walks through each defense layer.

A dedicated SERP endpoint returns parsed results; a general page API returns HTML you parse yourself. Domain surcharges on general APIs add up: ScraperAPI's pricing FAQ lists Google and Bing at 25 credits per request. We compare the specialist options in our review of the best SERP APIs.

Dedicated record APIs (WebScrapingAPI's Data API with published Amazon plans; Amazon targets on Oxylabs, Scrapingdog, and ScraperAPI) usually beat scraping product HTML yourself, because the provider maintains the parsers. For long-tail stores, use a general scraping API with rendering and geotargeting plus your own extraction rules. To see what building it yourself involves, follow our hands-on guide to scraping Amazon product data with Python.

Markdown cuts boilerplate for RAG and agents; HTML or JSON is better when you need exact fields such as prices. Firecrawl, Decodo, and ZenRows document markdown output. WebScrapingAPI returns HTML, a JSON envelope, or selected fields, which you can convert yourself when markdown is only a convenience. We cover LLM-based extractors separately in our list of AI web scraping tools.

Crawl and map endpoints (Firecrawl, Decodo, and WebScrapingAPI's Crawl API, which is scoped by quote) discover and fetch many URLs in one workflow. The alternative is your own crawler calling a page API per URL. Respect the Robots Exclusion Protocol (RFC 9309) either way.

Maintained record APIs (WebScrapingAPI's Data API lists 116 supported sources; Zyte and Oxylabs offer AI parsing) shift parser maintenance to the vendor. Your own selectors and extraction rules keep full control but own the breakage when markup changes. Who owns that maintenance is the real decision.

We read each provider's public pricing page and documentation and compared the billing unit, multiplier rules, failed-request billing, output formats, and specialist endpoints. We do not publish success-rate rankings, because results swing by target and week; run the test below on your own URLs instead. WebScrapingAPI is our product; every other provider is named from its own public pages.

Prices are as listed at the time of writing. Plans change often, so confirm on each provider's site before you buy.

WebScrapingAPI is our product: a ladder from Residential, Datacenter, and ISP proxies through Scraper, Browser, SERP, Crawl, and Data APIs to Data Feeds, Data Marketplace, Web Archive, and Managed Data. It suits teams that want page collection, search results, Amazon records, and proxies under one vendor, with a published Scraper API trial of 5,000 requests—not markdown out of the box or a marketplace of community-built scrapers.

Scraper API Starter is $19/mo for 7,000 requests (Basic $49/20,000; up to Pro $599/450,000). Every Scraper API plan includes JavaScript rendering, all-country geotargeting, proxy rotation and anti-bot handling, and auto parsing; rendering and geotargeting cost no extra credits, and failed Scraper API requests aren't billed. SERP API Google plans start at $28/mo; Amazon Data API plans start at $59/mo; Managed Data programs start at $299/mo. Browser, Crawl, Datacenter, and ISP products are quote-based. The Scraper API returns a page, a JSON envelope, or selected fields from a single request, and docs expose per-request cost plus WSA-Call-Credit-Usage headers. Compare static vs render_js=1 on your URLs and read the credit headers.

Bright Data's Web Unlocker is a request-priced unlocking API aimed at blocked targets, with proxy infrastructure behind it. Best for pay-per-success unlocking with unlimited concurrency on published Unlocker plans; not ideal for a simple credit-based page API without Unlocker-style packaging.

Pay as you go is $1.50 per 1,000 requests; Scale is $499/mo with 383,000 requests then $1.30 per 1,000; free tier 5,000 requests/mo. The pricing page states pay only for success, and notes that USD pricing is binding when local-currency estimates appear.

Oxylabs sells a Web API subscription that covers scrape and search products with per-target pricing per 1,000 results. Best for one subscription across scrape/search targets with clear per-site rates; not for a single flat credit for every domain.

Starter $30/mo, Basic $100/mo, Flexible $500/mo, Enterprise from $2,500/mo, plus a $5 welcome credit. Entry-tier examples on the pricing page include /Scrape from $0.50 per 1,000 results, Google JS at $1.35, Google non-JS at $1.00, and AI parsing at $3.00 per 1,000 parsed pages. Price your actual target mix against the per-1K rows for those domains.

Zyte API prices sites in five difficulty tiers for HTTP and browser-rendered responses, with pay-as-you-go and monthly commitments. Best for workloads where site difficulty varies and you want automated HTTP vs browser choice; not for one fixed credit price regardless of domain.

Try with $5 free credit (30 days). PAYG HTTP responses run from $0.13 to $1.27 per 1,000; browser-rendered from $1.01 to $16.08 per 1,000. A $100/mo commitment shows HTTP from $0.10 to $0.95 and browser-rendered from $0.75 to $12.00 per 1,000 on the pricing page. Billing is for successful responses; the page also states pricing from $0.06 per 1,000 successful responses at commitment.

ScrapingBee is a credit-based scraping API that renders JavaScript by default and publishes a clear credit table. Best for screenshots, extraction rules, and a documented credit ladder; not for high-volume static HTML if you forget to disable rendering.

Freelance $49/mo for 250,000 credits (50 concurrent); Startup $99/1M; Business $249/3M; Business+ $599/8M; 1,000 free credits. Classic proxy without JS costs 1 credit; JS rendering (default) costs 5; premium proxy 10 without JS / 25 with JS; stealth with JS costs 75. In auto mode, if every configuration fails, the request costs 0 credits. Prices exclude VAT.

ScraperAPI is a credit-based scraping API with domain surcharges and structured data products for popular sites. Best for teams that already budget Google, Amazon, or LinkedIn as expensive domains; not for global geotargeting on the Hobby plan (US and EU only).

Hobby $49/mo for 100,000 credits (20 threads); Startup $149/1M; Business $299/3M; Scaling $475/5M. Free plan with 1,000 credits; 7-day trial with 5,000 credits. Standard pages cost 1 credit; Amazon 5; Google and Bing 25; LinkedIn 30. Credits reset at renewal.

Firecrawl focuses on scrape, crawl, and map workflows with markdown-oriented output for AI pipelines. Best for LLM and RAG teams that want markdown and site-wide crawl/map endpoints; not for teams that mainly need SERP or Amazon record APIs.

Hobby $19/mo billed monthly ($16/mo billed annually) for 5,000 credits; Standard $83/mo billed annually for 100,000 credits; Growth $333; Scale $599; free 1,000 credits/mo. One credit per page for scrape/crawl/map; search costs 2 credits per 10 results; Interact costs 2 credits per browser minute; JSON, Question, or Highlight formats add 4 credits per page. A scrape that returns no result is not charged; 403 or 404 pages cost 1 credit. Credit rules are marked effective September 4, 2026. Compare markdown-only scrapes vs JSON-format scrapes (+4 credits) on the same URLs.

Decodo's web scraping API sells page and crawl allowances with markdown support and optional JS or premium proxies. Best for markdown plus scrape/search/map/crawl at a low entry price; not for buyers who need a detailed public credit matrix for every feature combination.

Free plan up to 10,000 pages or 5,000 crawls; $19/mo up to 38,000 pages or 19,000 crawls; $49/122,000 pages; $99/330,000; $249/1.2M. The pricing page says you only pay for what you use and that JS rendering and premium proxies can be enabled when needed.

ZenRows is a credit-based platform with fetch, extract, batch, and browser sessions on one balance, including markdown output. Best for a low entry plan and markdown without juggling separate products; not for SERP-specific published Google plans.

Build $16/mo for 45,000 credits (monthly pricing on the pricing page); Launch $58/250K; Growth $166/1.2M; Scale $458/5M; free 5,000 credits/mo. Standard page 1 credit; JS 5; premium proxies 10; JS + premium 25. Failed requests are never charged; 404 and 410 count as successful. Top-ups with rollover are listed on paid plans.

Scrapingdog offers credit plans across general, SERP, and ecommerce APIs, with failed requests refunded. Best for simple tiers and explicit "failed requests never charged" billing; not for a hosted browser action API as the primary interface.

Lite $40/mo for 200,000 credits; Standard $90/1M; Pro $200/3M; Premium $350/6M; free 100 credits. A standard request costs 1 credit; JavaScript rendering 5; premium proxy 10; JS + premium 25.

Scrape.do sells plans in successful API credits and states that features are unlocked across plans. Best for residential and browser capabilities while paying only for successful, valid content; not for buyers who need a public per-feature credit table before modeling cost.

Hobby $29/mo for 250,000 successful credits; Pro $99/1.25M; Business $249/3.5M; Advanced $699/10M; free 1,000 successful credits. The pricing page says you are charged only for successful responses with valid content. Define "valid content" checks on your side and compare billed successes to raw HTTP 200s.

Apify is a platform for running prebuilt or custom scrapers ("Actors") with compute-unit billing, rather than a single scraping endpoint. Best for a marketplace of scrapers or hosting your own scraping code; not for a single URL-in, HTML-out API without Actor selection.

Starter $19/mo plus pay as you go; Scale $199; Business $999; free $5 platform usage per month. Compute units are $0.20 / $0.16 / $0.13 by plan; Store Actors set their own prices.

The real cost is the price per credit × credits a request uses on your target ÷ the usable-data share, plus billed failures. Rendering, premium proxies, and hard domains multiply the base; on credit-based plans one request can use 1 to 75 credits.

Providers bill the same "request" differently:

A plan that looks cheaper per 1,000 credits can cost more per usable page once rendering and domain rules apply.

cost per usable page = (plan price ÷ included credits) × credits per request × (billed attempts ÷ usable results)

Three billing patterns show up on public pricing pages:

1. Only successes billed: Bright Data Web Unlocker, Scrapingdog, ZenRows, Scrape.do, and Zyte describe success-based billing.
2. Some error pages billed: Firecrawl charges 1 credit for 403 or 404 pages; a scrape with no result is not charged.
3. Every request billed: WebScrapingAPI's Browser API docs state that the product charges 100% of requests, successful and failed.

Watch the false-success trap: a block page with HTTP 200 that still consumes credits. Retries multiply that cost; use timeouts, retries, and backoff with jitter, and treat HTTP 429 Too Many Requests as a signal to slow down rather than hammer the same URL.

Worked example with list prices: ScrapingBee Freelance at $49 for 250,000 credits is $0.196 per 1,000 credits. Rendered requests at 5 credits cost about $0.98 per 1,000 attempts; premium proxy plus rendering at 25 credits about $4.90 per 1,000 attempts, before usable rate and failed-request rules.

def cost_per_1k_usable(
    plan_price: float,
    included_credits: int,
    credits_per_request: float,
    usable_rate: float,
    billed_rate: float,
) -> float:
    """Dollars per 1,000 usable pages for a credit-based plan."""
    price_per_credit = plan_price / included_credits
    cost_per_attempt = price_per_credit * credits_per_request * billed_rate
    return (cost_per_attempt / usable_rate) * 1000


# Example plan: replace with your own plan's price and included credits
plan_price, included = 100.0, 1_000_000
scenarios = [
    ("Static (1 credit)", 1, 0.95, 1.0),
    ("Rendered (5 credits)", 5, 0.90, 1.0),
    ("Rendered + premium (25 credits)", 25, 0.80, 1.0),
]
for name, credits, usable, billed in scenarios:
    cost = cost_per_1k_usable(plan_price, included, credits, usable, billed)
    print(f"{name}: ${cost:.2f} per 1,000 usable pages")

Concurrency caps rise with plan tier. Credits may reset monthly (ScraperAPI) or allow top-ups with rollover (ZenRows lists top-ups; Firecrawl lists rollover). Geotargeting can be gated by plan (ScraperAPI Hobby and Startup list US and EU only). Engineering time for parsing, monitoring, and data validation often exceeds the API invoice on hard targets.

Pick 50–200 real URLs per target type, run them through each API with identical settings, validate the content (not just the status code), and record usable-success rate, latency percentiles, credits used, and cost per usable page. Repeat on a second day before deciding.

1. Build a target list. CSV columns: url, target_type, must_contain (a text marker, or a CSS class written as .price, that a usable page must contain).
2. Keep keys in the environment. Read WSA_API_KEY (and other providers' keys) from env vars. If WebScrapingAPI is on your shortlist, our guide to making your first request with WebScrapingAPI gets you set up.
3. Cap concurrency to each plan's limit.
4. Validate content. Require HTTP 200, the marker present, and none of the common block strings (captcha, access denied, verify you are human).
5. Record credits. Prefer provider headers or envelope fields when available (WebScrapingAPI returns WSA-Call-Credit-Usage and cost in the JSON envelope).
6. Summarize usable rate, p50/p95 latency, credits used, and cost per 1,000 usable pages.
7. Repeat on another day before you choose.

import csv
import os
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from html.parser import HTMLParser
from urllib.parse import urlencode

import requests

BLOCK_MARKERS = ("captcha", "access denied", "verify you are human")
MAX_WORKERS = 4


def adapter_wsa(url: str, render_js: int = 0) -> requests.Response:
    api_key = os.environ["WSA_API_KEY"]
    params = {"api_key": api_key, "url": url, "render_js": render_js}
    return requests.get(
        "https://api.webscrapingapi.com/v2?" + urlencode(params),
        timeout=30,
    )


def adapter_plain(url: str, **_kwargs) -> requests.Response:
    """Local stand-in so you can verify the harness without a provider key."""
    return requests.get(url, timeout=30, headers={"User-Agent": "wsa-harness/1.0"})


class ClassCollector(HTMLParser):
    """Collects every class name used on any element of the page."""

    def __init__(self) -> None:
        super().__init__()
        self.classes: set[str] = set()

    def handle_starttag(self, tag, attrs) -> None:
        for name, value in attrs:
            if name == "class" and value:
                self.classes.update(value.split())


def has_css_class(body: str, css_class: str) -> bool:
    collector = ClassCollector()
    collector.feed(body)
    return css_class in collector.classes


def is_usable(status: int, body: str, must_contain: str) -> bool:
    if status != 200:
        return False
    lower = body.lower()
    if any(marker in lower for marker in BLOCK_MARKERS):
        return False
    if must_contain.startswith("."):
        return has_css_class(body, must_contain[1:])
    return must_contain.lower() in lower


def run_row(row: dict, adapter) -> dict:
    started = time.perf_counter()
    response = adapter(row["url"])
    body = response.text or ""
    return {
        "url": row["url"],
        "status": response.status_code,
        "latency_s": time.perf_counter() - started,
        "credits": response.headers.get("WSA-Call-Credit-Usage"),
        "usable": is_usable(response.status_code, body, row["must_contain"]),
    }


def main(path: str = "targets.csv") -> None:
    with open(path, newline="", encoding="utf-8") as handle:
        rows = list(csv.DictReader(handle))
    adapter = adapter_wsa if os.environ.get("WSA_API_KEY") else adapter_plain
    results = []
    with ThreadPoolExecutor(max_workers=MAX_WORKERS) as pool:
        futures = [pool.submit(run_row, row, adapter) for row in rows]
        for future in as_completed(futures):
            results.append(future.result())
    usable = sum(1 for item in results if item["usable"])
    print(f"usable={usable}/{len(results)}")
    for item in results:
        print(item)


if __name__ == "__main__":
    main()

Add other providers the same way: one adapter function each, built from that provider's docs. Free allowances for a first pass include WebScrapingAPI's 5,000 Scraper API requests, Firecrawl's 1,000 credits/mo, ZenRows' 5,000 credits/mo, ScrapingBee's 1,000 credits, Scrape.do's 1,000 successful credits, Bright Data's 5,000 Unlocker requests/mo.

Build when targets are few, open, and stable and you have people to maintain proxies, browsers, and parsers. Buy when targets are protected, varied, or business-critical, or when engineering time costs more than the API. Many teams combine both: their own crawler, with an API for the hard pages.

We break down the advantages and tradeoffs of a web scraping API responsibility by responsibility.

Use proxies when your own client or browser should control requests, sessions, retries, and parsing. Use an API when you want the provider to run routing, rendering, and block handling and hand back a response. Proxies are billed by traffic (GB); APIs by request.

The unit difference is concrete on WebScrapingAPI's published pages: Residential Proxies pay as you go from $8.50/GB, while Scraper API plans are counted in API calls starting at $19/mo for 7,000 requests. If you go the proxy route, our guide to choosing between datacenter, residential, and ISP proxies explains the tradeoffs.

Buy data when you need the same fields from the same sources on a schedule, when parser maintenance is not your core work, or when you need history. Options run from ready-made datasets to scheduled feeds to a fully managed scraping service with agreed schemas and delivery.

WebScrapingAPI offers Data Marketplace, Data Feeds, Web Archive, and Managed Data (programs start at $299 per month). Bright Data and Oxylabs also sell datasets alongside their APIs. Buying data is often the right answer when you care about delivery rather than operating scrapers.

Using an API does not change the legal picture. What matters is which data you collect and how you use it. Stick to public data, respect site terms and robots.txt, avoid personal data unless you have a lawful basis, and get legal advice for your jurisdiction. The API is a tool; responsibility stays with you.

For a practical overview of robots.txt, see Google's introduction to robots.txt. This section is not legal advice.

There's no single best web scraping API. The right choice depends on the job: static pages, JavaScript-heavy apps, protected sites, search results, ecommerce records, or markdown for LLMs. Shortlist two or three, compare cost per successful page, and test on your own URLs.

A hosted service that fetches a page or search result through a single HTTP request. The provider runs proxies, retries, browsers, and block handling, then returns HTML, JSON, or selected fields. You pay per request or credit instead of running infrastructure.

Most entry plans in this guide cost about $16 to $49 a month; several offer pay-as-you-go, and most have free allowances. Credits per request rise for rendering, premium proxies, and hard domains, and failed-request billing decides the real cost. Compare cost per usable page, not plan price.

Hard targets need residential or premium proxies, a real browser, more retries, and CAPTCHA handling. Providers pass that on as credit multipliers or per-site tiers; one request can cost several to dozens of times a basic one.

It depends on the provider and product. Many bill only successes, some bill certain error pages, and some (especially hosted-browser products) bill every request. A block page returned with HTTP 200 may also be billed as a success unless the provider detects it.

Neither is better in general. Proxies suit teams that want their own client in control and are billed by traffic; an API runs routing, rendering, and block handling for you and bills per request. Many teams use both.

Build when targets are few, unprotected, and stable and you have engineers to maintain proxies, browsers, and parsers. Use an API when targets are protected or varied, when you need data quickly, or when maintenance costs more than API usage. Hybrids are common.

Choose an API that renders JavaScript on request and, if content appears only after clicks or scrolling, one with a hosted browser that accepts waits and actions. Rendering often costs several times a basic request, so turn it off for pages that don't need it.

For major marketplaces such as Amazon, a dedicated record API that returns parsed product, price, and review fields usually beats scraping raw HTML. For long-tail stores, use a general scraping API with rendering and geotargeting plus your own extraction rules.

Firecrawl, Decodo, and ZenRows document markdown output. Others, including WebScrapingAPI, return HTML, JSON, or selected fields you can convert yourself. Choose structured JSON when you need exact fields such as prices.

Using an API doesn't change the legal position. Collect only publicly available data, follow site terms and robots.txt, avoid personal data without a lawful basis, and get legal advice for your jurisdiction.

Use the free allowance on 50 to 200 of your own URLs per target type with identical settings. Check that the content you need is there, record usable-success rate, latency, and credits, divide the cost by usable pages, and repeat on another day before choosing.

Pick your hardest URL, run the harness on two or three shortlisted APIs (including our 5,000-request Scraper API trial), and compare cost per usable page the same day. The provider that wins on your targets is the one to choose.