YepAPI
Web Scraping

Web Scraping API

Web Scraping API — scrape any URL to clean Markdown, HTML, or plain text. Smart mode escalates to JS rendering and premium proxies automatically, one flat $0.01 per call.

Pricing, live playground and use cases: Web Scraping API on YepAPI

POST/v1/scrape
$0.01/call

The Web Scraping API fetches any public URL and returns the page as clean Markdown (ideal for LLMs and RAG), raw HTML, or plain text. It runs in smart mode: each request starts with the cheapest fetch that can work and automatically escalates — to a headless browser with JavaScript rendering, then to premium residential proxies — until the page loads. You always pay one flat $0.01, whatever it took.

Need to click, scroll, or wait for an element before the page is captured? Use the JavaScript Scrape API. For sites behind heavy anti-bot protection, use the Stealth Scrape API.

Usage

const res = await fetch('https://api.yepapi.com/v1/scrape', {
  method: 'POST',
  headers: {
    'x-api-key': 'YOUR_API_KEY',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ url: 'https://example.com', format: 'markdown' }),
});
const { data } = await res.json();
console.log(data.content);
curl -X POST https://api.yepapi.com/v1/scrape \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "format": "markdown"}'
import requests

res = requests.post(
    "https://api.yepapi.com/v1/scrape",
    headers={"x-api-key": "YOUR_API_KEY"},
    json={"url": "https://example.com", "format": "markdown"},
)
print(res.json()["data"]["content"])

Request Body

ParameterTypeRequiredDescriptionDefault
urlstringYesFull http(s) URL of the page to scrape (max 4096 characters). Localhost and private IPs are rejected—
formatstringNoOutput format. One of: markdown, html, textmarkdown
selectorstringNoCSS or XPath selector. Returns only the first matching element — its HTML when format is html, otherwise its text (max 500 characters)—
wait_fornumberNoExtra milliseconds to wait after the page loads, max 15000—

waitFor is accepted as an alias of wait_for.

Info

Browser-only options — js_scenario, wait_for_selector, device, and block_ads — are not accepted here because smart mode may answer from a plain HTTP fetch. Sending one returns a 400 that points you to /v1/scrape/js. country is only accepted on /v1/scrape/stealth.

Response

{
  "ok": true,
  "data": {
    "url": "https://example.com/",
    "statusCode": 200,
    "content": "# Example Domain\n\nThis domain is for use in documentation examples without needing permission. Avoid use in operations.\n\n[Learn more](https://iana.org/domains/example)",
    "format": "markdown"
  }
}

Response Fields

FieldTypeDescription
okbooleanWhether the request succeeded
data.urlstringFinal URL after redirects
data.statusCodenumberHTTP status code the target page returned
data.contentstringThe page (or the selector match) in the requested format. Empty string if selector matched nothing
data.formatstringThe format used: markdown, html, or text

Smart Mode

/v1/scrape picks the cheapest way to load each page:

  1. A plain HTTP fetch — fast, and enough for most blogs, docs, and static pages.
  2. If the page needs JavaScript or the fetch is blocked, a headless Chrome render.
  3. If it is still blocked, premium residential proxies.

Escalation happens inside the same request; the price stays $0.01. If the site still refuses every attempt, you get a 502 UPSTREAM_ERROR and are not charged — retry, or switch to Stealth Scrape for heavily protected sites.

Caching

Results are cached for 1 hour per URL and option set. Repeating the same request within that window returns the cached page.

Errors

StatusCodeWhen
400VALIDATION_ERRORurl is missing, not a full http(s) URL, or points to localhost / a private IP; format is invalid; wait_for is over 15000; a browser-only option or country was sent; or the URL is a Google page (use /v1/serp/google for Google results)
404NOT_FOUNDThe target page returned 404 Not Found
429RATE_LIMITED_UPSTREAMToo many scrapes in flight — retry in a few seconds
502UPSTREAM_ERRORThe site blocked the request or failed to load — retry, or use /v1/scrape/stealth
503UPSTREAM_TIMEOUTThe page took too long to load (about 25 seconds) — retry

Failed calls are not charged.

On this page