How to Use cURL With Python in 2026

If you write Python and consume HTTP APIs, you have probably hit this moment: an API doc or your browser's "Copy as cURL" button hands you a one-liner starting with curl -X POST ..., and now you need it inside a Python script. Figuring out how to use cURL with Python sounds simple, but has more than one right answer.

cURL itself is a command-line tool for transferring data over network protocols (HTTP, HTTPS, FTP). From Python, you can call the curl binary as an external process, drive its underlying C library (libcurl) through PycURL, or use the Requests library as a Pythonic alternative. Each option has trade-offs around speed, control, and maintainability.

This guide is for backend, data, and scraping engineers who already know Python and want a clean way to translate any curl snippet into working code. We cover all three methods with runnable examples, map common curl flags to Python equivalents, build a small scraping pipeline, and finish with troubleshooting so you can ship the code instead of fighting your tooling.

Most teams hit the curl-in-Python question for the same reason: someone handed them a curl command. API docs ship request examples as curl invocations, browser devtools export network calls in the same format, and Postman and Insomnia let you copy any request as curl. That snippet is the source of truth, and you want your Python code to behave identically.

Running curl inside Python lets you debug the exact request first, then graduate to something more idiomatic. A copy-paste run via subprocess proves the endpoint works. From there, you can rewrite the call in PycURL or Requests with confidence that you have not changed the wire format. That short feedback loop is the real reason developers want to know how to use cURL with Python, not curl on its own.

When developers ask how to use cURL with Python, they usually mean one of three concrete approaches. Before you write any code, pick the one that matches the job. The three options are not interchangeable, and choosing the wrong one usually means rewriting the call later.

Treat subprocess as your translation step, PycURL as your power tool, and Requests as your default. Most production Python code ends up on Requests; the other two cover the edges.

The subprocess module is part of Python's standard library, so you can shell out to curl without installing anything new. This is the most literal reading of "use cURL with Python," and it is genuinely useful when you want to replay a command exactly as it appears in API docs.

One non-negotiable safety rule: pass the command as a list of arguments, not a single shell string. Strings invite shell injection when any part of the request comes from user input. The arg-list form skips the shell entirely. The Python subprocess documentation covers the security model in detail.

Take a curl one-liner, split it into tokens, and hand the list to subprocess.run. Set capture_output=True so stdout and stderr come back to you, and text=True so you get strings instead of bytes.

import subprocess

cmd = [
    "curl", "-s",
    "-H", "Accept: application/json",
    "https://httpbin.org/get?lang=python",
]

result = subprocess.run(cmd, capture_output=True, text=True, timeout=15)
print(result.stdout)

The -s flag silences curl's progress meter so stdout contains only the response body. The timeout=15 argument raises subprocess.TimeoutExpired if curl hangs, which is exactly what you want in a script that should not block forever. Keep this form for translation: once it works, you have a verified baseline to port to PycURL or Requests.

By default, subprocess.run does not raise when curl exits non-zero. You have to inspect the return code yourself, or opt into an exception explicitly.

import json, subprocess

result = subprocess.run(cmd, capture_output=True, text=True, timeout=15)

if result.returncode != 0:
    raise RuntimeError(f"curl failed ({result.returncode}): {result.stderr.strip()}")

try:
    payload = json.loads(result.stdout)
except json.JSONDecodeError as exc:
    raise RuntimeError(f"non-JSON response: {result.stdout[:200]}") from exc

print(payload["args"])

You can also call result.check_returncode(), which raises CalledProcessError on any non-zero exit. Either way, log result.stderr when the call fails. Curl writes its diagnostic output there, and that message is usually enough to distinguish a DNS failure from a TLS error or a 4xx response.

PycURL is a Python interface to libcurl, the same C library that powers the curl binary itself. It exposes low-level options for timeouts, SSL configuration, headers, cookies, redirects, and protocols beyond HTTP. When throughput or fine-grained control matters, PycURL is the right call.

Install it with pip install pycurl. The Python package is a thin wrapper, so you also need libcurl and OpenSSL development headers on the system. On Debian/Ubuntu that is apt install libcurl4-openssl-dev libssl-dev; on macOS, brew install curl openssl. We will cover the OpenSSL link-time errors in the troubleshooting section, because they are the most common reason a fresh install fails.

PycURL follows the libcurl pattern: create a handle, set options, perform, then close. Writes go to a file-like buffer, which is usually a BytesIO.

import json, pycurl
from io import BytesIO
from urllib.parse import urlencode

# GET
buf = BytesIO()
c = pycurl.Curl()
c.setopt(c.URL, "https://httpbin.org/get?lang=python")
c.setopt(c.WRITEDATA, buf)
c.perform()
status = c.getinfo(pycurl.RESPONSE_CODE)
c.close()
print(status, buf.getvalue().decode("utf-8"))

For a form-encoded POST, set POSTFIELDS to a urlencoded body. For JSON, dump the dict and set the right Content-Type.

# POST form
form = urlencode({"a": 1, "b": "two"})
buf = BytesIO()
c = pycurl.Curl()
c.setopt(c.URL, "https://httpbin.org/post")
c.setopt(c.POSTFIELDS, form)
c.setopt(c.WRITEDATA, buf)
c.perform(); c.close()

# POST JSON
body = json.dumps({"hello": "world"})
buf = BytesIO()
c = pycurl.Curl()
c.setopt(c.URL, "https://httpbin.org/post")
c.setopt(c.HTTPHEADER, ["Content-Type: application/json"])
c.setopt(c.POSTFIELDS, body)
c.setopt(c.WRITEDATA, buf)
c.perform(); c.close()

Setting HTTPHEADER is how you control Content-Type, Accept, Authorization, and any other request header. We will build on that pattern next.

HTTPHEADER takes a list of "Name: value" strings. Cookies can ride along as a Cookie header for one-shot calls, or you can let libcurl manage a cookie jar with COOKIEFILE and COOKIEJAR.

c.setopt(c.HTTPHEADER, [
    "Accept: application/json",
    "Authorization: Bearer eyJhbGciOi...",
    "Cookie: session=abc123; theme=dark",
])

# Or use a cookie jar to persist Set-Cookie across requests
c.setopt(c.COOKIEFILE, "cookies.txt")
c.setopt(c.COOKIEJAR, "cookies.txt")

For redirects, enable FOLLOWLOCATION (the equivalent of curl -L) and cap the chain with MAXREDIRS so a misbehaving server cannot loop you forever.

c.setopt(c.FOLLOWLOCATION, True)
c.setopt(c.MAXREDIRS, 5)

If you only need response headers (a curl -I style request), set NOBODY to True and route the header stream to a callback via HEADERFUNCTION. That callback runs once per header line, which is handy when scraping for things like Last-Modified or rate-limit metadata. For deeper recipes, see our breakdown of HTTP response headers in cURL.

WRITEDATA accepts any file-like object, so a download is a one-line change: open a file in binary write mode and point libcurl at it. Memory usage stays flat regardless of payload size.

import os, pycurl

url = "https://example.com/large.iso"
out = "large.iso"

mode = "ab" if os.path.exists(out) else "wb"
offset = os.path.getsize(out) if mode == "ab" else 0

with open(out, mode) as fp:
    c = pycurl.Curl()
    c.setopt(c.URL, url)
    c.setopt(c.WRITEDATA, fp)
    c.setopt(c.FOLLOWLOCATION, True)
    if offset:
        c.setopt(c.RANGE, f"{offset}-")  # resume from byte offset
    c.perform(); c.close()

The Range: bytes={offset}- header tells the server to send only the missing tail, which is exactly how curl -C - resumes interrupted downloads. The server must support range requests (most CDNs do).

For most everyday work, Requests is the answer. It is not bundled with Python (install with pip install requests), but the API maps cleanly to curl semantics: query params, headers, cookies, JSON bodies, and timeouts are all keyword arguments.

import requests

# GET with query params
r = requests.get(
    "https://httpbin.org/get",
    params={"lang": "python"},
    headers={"Accept": "application/json"},
    timeout=15,
)
r.raise_for_status()
print(r.json())

# POST JSON
r = requests.post(
    "https://httpbin.org/post",
    json={"hello": "world"},
    headers={"Authorization": "Bearer ..."},
    timeout=15,
)

raise_for_status() is your friend: it turns any 4xx/5xx into a requests.HTTPError, so the difference between network failures (requests.ConnectionError, requests.Timeout) and HTTP errors stays clear in your code.

Pick Requests by default when the call is one of many in a Python program, you need session state, or your team will maintain the code. Pick PycURL when you have measured a real bottleneck or need libcurl-only options. For a deeper comparison of Python HTTP clients, see our roundup on the topic.

Once you know how to use cURL with Python in all three flavors, the fastest way to port a new command is to translate its flags one at a time. This table covers the flags you will see in 95% of API docs and devtools exports.

If you would rather not translate by hand, there are public curl-to-Python converters (and ScrapingBee's converter is a well-known one) that take a curl command and output a Requests call with headers, params, and data filled in. Use them to bootstrap, then sanity-check the output against this table.

A common reason to learn how to use cURL with Python is web scraping. Here is the pattern we use when prototyping a small scraper: fetch HTML with PycURL for speed, parse it with BeautifulSoup, then persist the result. The whole thing is under 40 lines and covers status checks, encoding, and a CSV writeout.

import csv, json, pycurl
from io import BytesIO
from bs4 import BeautifulSoup

URL = "https://books.toscrape.com/catalogue/page-1.html"

def fetch(url: str) -> str:
    buf = BytesIO()
    c = pycurl.Curl()
    c.setopt(c.URL, url)
    c.setopt(c.WRITEDATA, buf)
    c.setopt(c.FOLLOWLOCATION, True)
    c.setopt(c.TIMEOUT, 20)
    c.setopt(c.HTTPHEADER, ["User-Agent: scraping-pipeline/1.0"])
    c.perform()
    code = c.getinfo(pycurl.RESPONSE_CODE)
    c.close()
    if code != 200:
        raise RuntimeError(f"unexpected status {code} for {url}")
    return buf.getvalue().decode("utf-8")

def parse(html: str) -> list[dict]:
    soup = BeautifulSoup(html, "html.parser")
    rows = []
    for card in soup.select("article.product_pod"):
        rows.append({
            "title": card.h3.a["title"],
            "price": card.select_one(".price_color").text.strip(),
            "stock": card.select_one(".availability").text.strip(),
        })
    return rows

def main():
    rows = parse(fetch(URL))
    with open("books.csv", "w", newline="") as fp:
        writer = csv.DictWriter(fp, fieldnames=rows[0].keys())
        writer.writeheader(); writer.writerows(rows)
    print(json.dumps(rows[:2], indent=2))

if __name__ == "__main__":
    main()

Swap PycURL for subprocess if you want to replay a curl command verbatim, or for Requests if you need session state. Once you start hitting real-world sites with rate limits and anti-bot defenses, you will want a proxy layer in front of this fetch step.

Most of the pain when learning how to use cURL with Python comes from a small set of recurring failures. Here is the short list and how to fix each one.

• PycURL ImportError: pycurl: libcurl link-time ssl backend (...) is different from compile-time ssl backend (...). Your PycURL wheel was built against a different SSL backend than the libcurl on your system. At the time of writing, the most reliable fix on macOS is to install OpenSSL through Homebrew and reinstall PycURL from source against it; on Windows, install OpenSSL 1.1.x binaries and set PYCURL_SSL_LIBRARY, LIB, and INCLUDE before pip install pycurl --no-binary :all:. Re-check the PycURL install notes for your platform because the exact env vars have shifted across releases.
• UnicodeEncodeError from PycURL on POST bodies. PycURL expects bytes for POSTFIELDS. Encode non-ASCII data explicitly: c.setopt(c.POSTFIELDS, body.encode("utf-8")).
• subprocess.TimeoutExpired. Always pass timeout= to subprocess.run. Treat the exception as a network failure, not a bug.
• TLS errors and self-signed certs. PycURL: c.setopt(c.SSL_VERIFYPEER, 0); Requests: verify=False. Only do this in trusted environments, and prefer pinning a CA bundle in production.
• Distinguishing HTTP vs transport errors. With Requests, catch requests.HTTPError separately from requests.ConnectionError. With subprocess, a non-zero returncode is a transport-level failure; an HTTP 4xx still exits 0 unless you pass --fail.

Once you have all three tools, the choice is situational.

• Debugging an API or reproducing a bug report. Reach for subprocess first. Running the literal curl command removes "my Python client is the problem" from the variables.
• One-shot scripts and CI jobs. Requests. It is readable, well-documented, and easy for the next engineer to maintain.
• Long-running scrapers, high request volumes, or protocols beyond HTTP. PycURL. You get libcurl's connection reuse, fine-grained TLS control, and lower per-request overhead.
• Cookies and login flows. Requests Session is the path of least resistance; PycURL's cookie jar is the alternative when you are already on libcurl.

Knowing how to use cURL with Python is less about picking a winner and more about matching the tool to the request.

• Three real options for combining curl with Python: subprocess (replay the binary), PycURL (libcurl bindings), and Requests. Pick deliberately, not by habit.
• Use subprocess as a translation layer: verify the wire format with the literal command, then port to PycURL or Requests with confidence.
• Map curl flags to Python systematically: -H becomes headers, -d becomes data or json, -G becomes params, -F becomes files, -L becomes follow-redirects, -o becomes a streamed write.
• Always set timeouts, always check status codes, and treat HTTP errors as a distinct category from transport errors.
• For high-volume scraping or downloads with resume support, PycURL pays for itself. For everything else, default to Requests.

Both. "Running curl" usually means calling the system curl executable from Python via subprocess, which requires curl on your PATH. "Using a Python library" means importing PycURL (a wrapper around libcurl) or Requests (a pure-Python HTTP client) and never touching the binary. The wire requests look identical to the server; only the calling code differs.

PycURL is generally faster in synthetic benchmarks because it offloads HTTP work to libcurl in C. In real workloads, the gap shrinks: network latency, TLS handshakes, and parsing usually dominate. PycURL still wins on thousands of concurrent connections, where per-request overhead compounds. For most scripts, the difference is not measurable.

Paste the curl command into a curl-to-Python converter (several free ones exist online), pick the Requests output, then review the generated code against the flag translation table in this guide. The converter handles headers, params, and data automatically. You still want to add timeout, raise_for_status(), and proper exception handling before shipping it.

In Requests, use the files keyword: requests.post(url, files={"upload": open("data.csv", "rb")}). For extra control over filename and content-type, pass a tuple: files={"upload": ("data.csv", fp, "text/csv")}. In PycURL, set the HTTPPOST option with a list of tuples describing each form field, including a pycurl.FORM_FILE entry for the path on disk.

That error means the wheel was built against an SSL backend that does not match your system libcurl. The fix is to reinstall PycURL from source against your local libcurl: pip install --no-binary :all: pycurl after installing libcurl and OpenSSL development headers (brew install curl openssl on macOS; the equivalent dev packages on Linux). On Windows, set the OpenSSL paths via environment variables before reinstalling.

Knowing how to use cURL with Python is really three skills in one: shelling out to the curl binary with subprocess, driving libcurl through PycURL, and writing idiomatic Requests code. Each has a job. subprocess is your translation step from API docs to verified code. PycURL is your performance tool for high-throughput scrapers and downloads. Requests is the default for everything else, because it stays readable as a project grows.

The flag-to-Python table, the streaming download recipe, and the troubleshooting list above cover most of the rough edges. The remaining one is what your target server does to non-browser traffic: rate limits, CAPTCHAs, JavaScript challenges, and IP blocks hit you eventually, no matter how clean your Python code is.

That is the layer we focus on at WebScrapingAPI. If you are spending more time fighting anti-bot defenses than writing your scraper, our Scraper API takes a curl-style request and returns the raw HTML, handling proxy rotation, CAPTCHA solving, and retries on its side, so you can keep your subprocess, PycURL, or Requests code exactly as it is and just swap the endpoint. Pick the curl method that fits your job, and let the network layer be someone else's problem.