YepAPI
Web Scraping

JavaScript Scraping API

JavaScript Scraping API — scrape JS-rendered pages with headless Chrome. Click, scroll, fill forms, wait for elements, and run custom JavaScript before capture. $0.02 per call.

Pricing, live playground and use cases: JavaScript Scraping API on YepAPI

POST/v1/scrape/js
$0.02/call

The JavaScript Scraping API always loads the page in headless Chrome, so content rendered client-side by React, Vue, Angular, or any other framework is there when the page is captured. On top of rendering, you can script the browser with js_scenario — click "load more", scroll, fill a search box, wait for an element, or run your own JavaScript and get its return value — then receive the page as Markdown, HTML, or text.

For pages that do not need a browser, the Web Scraping API is cheaper. If the site blocks headless browsers, use Stealth Scrape.

Usage

const res = await fetch('https://api.yepapi.com/v1/scrape/js', {
  method: 'POST',
  headers: {
    'x-api-key': 'YOUR_API_KEY',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    url: 'https://example.com/products',
    format: 'markdown',
    wait_for_selector: '.product-card',
    js_scenario: {
      instructions: [
        { click: 'button.load-more' },
        { wait_for: '.product-card:nth-child(40)' },
        { scroll_y: 2000 },
        { wait: 1000 },
        { evaluate: 'document.querySelectorAll(".product-card").length' },
      ],
    },
  }),
});
const { data } = await res.json();
console.log(data.content);
console.log(data.evaluateResults); // ["40"]
curl -X POST https://api.yepapi.com/v1/scrape/js \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/products",
    "format": "markdown",
    "wait_for_selector": ".product-card",
    "js_scenario": {"instructions": [
      {"click": "button.load-more"},
      {"wait_for": ".product-card:nth-child(40)"},
      {"scroll_y": 2000},
      {"wait": 1000},
      {"evaluate": "document.querySelectorAll(\".product-card\").length"}
    ]}
  }'
import requests

res = requests.post(
    "https://api.yepapi.com/v1/scrape/js",
    headers={"x-api-key": "YOUR_API_KEY"},
    json={
        "url": "https://example.com/products",
        "format": "markdown",
        "wait_for_selector": ".product-card",
        "js_scenario": {"instructions": [
            {"click": "button.load-more"},
            {"wait_for": ".product-card:nth-child(40)"},
            {"scroll_y": 2000},
            {"wait": 1000},
            {"evaluate": "document.querySelectorAll('.product-card').length"},
        ]},
    },
)
data = res.json()["data"]
print(data["content"], data.get("evaluateResults"))

Request Body

ParameterTypeRequiredDescriptionDefault
urlstringYesFull http(s) URL of the page to scrape (max 4096 characters). Localhost and private IPs are rejected—
formatstringNoOutput format. One of: markdown, html, textmarkdown
selectorstringNoCSS or XPath selector. Returns only the first matching element — its HTML when format is html, otherwise its text (max 500 characters)—
wait_fornumberNoExtra milliseconds to wait after the page loads, max 15000—
wait_for_selectorstringNoCSS selector to wait for before capturing. Returns 400 if it never appears (max 500 characters)—
js_scenarioobjectNoBrowser actions to run before capture: {"instructions": [...]} or the bare array. See JS Scenario (max 8000 characters)—
devicestringNoBrowser profile. One of: desktop, mobiledesktop
block_adsbooleanNoBlock ads and trackers — faster loads and cleaner contentfalse

camelCase aliases are also accepted: waitFor, waitForSelector, jsScenario, blockAds. For backward compatibility, a non-numeric string in wait_for is treated as wait_for_selector.

JS Scenario

js_scenario is a list of instructions the browser runs in order after the page loads and before it is captured:

InstructionExampleWhat it does
click{"click": "#load-more"}Click the element matching a CSS selector
wait{"wait": 1000}Pause for a number of milliseconds
wait_for{"wait_for": ".results"}Wait until an element appears
wait_for_and_click{"wait_for_and_click": "#accept-cookies"}Wait for an element, then click it
scroll_x{"scroll_x": 1000}Scroll horizontally by N pixels
scroll_y{"scroll_y": 1000}Scroll vertically by N pixels
fill{"fill": ["#search", "running shoes"]}Type a value into an input: [selector, value]
evaluate{"evaluate": "document.title"}Run JavaScript; its result is returned in evaluateResults
infinite_scroll{"infinite_scroll": {"max_count": 0, "delay": 1000}}Keep scrolling to load lazy content. max_count is the number of scrolls (0 = until the page stops growing), delay the ms between scrolls

A typical scenario accepts the cookie banner, clicks "load more", waits for the new items, and reads a value with evaluate. An invalid instruction returns a 400 VALIDATION_ERROR naming the problem. The whole call has to finish within about 25 seconds, so keep waits short.

Response

{
  "ok": true,
  "data": {
    "url": "https://example.com/products",
    "statusCode": 200,
    "content": "# Products\n\n## Trail Runner 3\n$89.00\n\n## Road Racer Pro\n$129.00\n\n...",
    "format": "markdown",
    "evaluateResults": ["40"],
    "jsScenarioReport": {
      "total_duration": 3.48,
      "task_executed": 5,
      "task_success": 5,
      "task_failure": 0,
      "tasks": [
        { "task": "click", "params": "button.load-more", "duration": 0.21, "success": true },
        { "task": "wait_for", "params": ".product-card:nth-child(40)", "duration": 1.12, "success": true },
        { "task": "scroll_y", "params": 2000, "duration": 0.1, "success": true },
        { "task": "wait", "params": 1000, "duration": 1.0, "success": true },
        { "task": "evaluate", "params": "document.querySelectorAll(\".product-card\").length", "duration": 0.02, "success": true }
      ]
    }
  }
}

Response Fields

FieldTypeDescription
okbooleanWhether the request succeeded
data.urlstringFinal URL after redirects
data.statusCodenumberHTTP status code the target page returned
data.contentstringThe page (or the selector match) in the requested format, captured after the scenario ran. Empty string if selector matched nothing
data.formatstringThe format used: markdown, html, or text
data.evaluateResultsarrayOne result per evaluate instruction, in order. Only present when the scenario has evaluate steps
data.jsScenarioReportobjectPer-step report: total_duration (seconds), task_executed, task_success, task_failure, and tasks[] with each step's task, params, duration, and success. Only present when js_scenario was sent

Caching

Results are cached for 1 hour per URL and option set (including the scenario). Repeating the same request within that window returns the cached page.

Errors

StatusCodeWhen
400VALIDATION_ERRORurl is missing, invalid, or private; format / device is invalid; wait_for is over 15000; js_scenario is malformed, too long, or has an invalid instruction; wait_for_selector never matched an element; country was sent (stealth only); or the URL is a Google page (use /v1/serp/google)
404NOT_FOUNDThe target page returned 404 Not Found
429RATE_LIMITED_UPSTREAMToo many scrapes in flight — retry in a few seconds
502UPSTREAM_ERRORThe site blocked the request or failed to load — retry, or use /v1/scrape/stealth
503UPSTREAM_TIMEOUTThe page (plus scenario) took too long — about 25 seconds

Failed calls are not charged.

On this page