YepAPI
Web Scraping

AI Scraping API

AI Scraping API — extract structured JSON from any webpage with a plain-English prompt or a typed schema. No CSS selectors. $0.03 per call.

Pricing, live playground and use cases: AI Scraping API on YepAPI

POST/v1/scrape/ai-extract
$0.03/call

The AI Scraping API renders a page in a headless browser and has an AI model pull out the data you describe — no CSS selectors or XPath to write or maintain when the site's markup changes. Describe what you want with prompt, or pass a schema to get back an object with exactly the fields and types you defined.

For selector-based extraction at $0.01, use the Data Extraction API.

Usage

With a typed schema — data.extracted has exactly these fields:

const res = await fetch('https://api.yepapi.com/v1/scrape/ai-extract', {
  method: 'POST',
  headers: {
    'x-api-key': 'YOUR_API_KEY',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    url: 'https://example.com/product/123',
    schema: {
      type: 'object',
      properties: {
        title: { type: 'string', description: 'Product name' },
        price: { type: 'number', description: 'Current sale price' },
        in_stock: { type: 'boolean', description: 'Whether it can be bought now' },
      },
    },
  }),
});
const { data } = await res.json();
console.log(data.extracted); // { title, price, in_stock }
curl -X POST https://api.yepapi.com/v1/scrape/ai-extract \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/product/123", "schema": {"title": "the product name", "price": "price in USD", "in_stock": "whether it is in stock"}}'

With only a prompt:

import requests

res = requests.post(
    "https://api.yepapi.com/v1/scrape/ai-extract",
    headers={"x-api-key": "YOUR_API_KEY"},
    json={
        "url": "https://example.com/product/123",
        "prompt": "Return JSON with the product name, price, and a one-sentence description",
    },
)
print(res.json()["data"]["extracted"])

Request Body

ParameterTypeRequiredDescriptionDefault
urlstringYesFull http(s) URL of the page (max 4096 characters)—
promptstringNoWhat to extract, in plain English (max 2000 characters). Required unless schema is given; with a schema it is used as extra context—
schemaobjectNoFields to extract: a JSON Schema with properties, or a flat {"field": "description"} map—

Prompt vs. Schema

  • Prompt only: data.extracted is the AI's answer. When the answer is structured — ask for JSON to get JSON — it is a JSON object or array; otherwise it is a string.
  • Schema: data.extracted is an object with exactly the fields you listed. Two shapes are accepted:
    • JSON Schema: {"type": "object", "properties": {"price": {"type": "number", "description": "Sale price"}, "title": {"type": "string"}}}. Supported types: string, number (integer is treated as number), boolean, array, object. description tells the model what each field means; without it the field name is used.
    • Flat map: {"title": "the product name", "price": "price in USD"} — each value is the field's description.

Response

{
  "ok": true,
  "data": {
    "url": "https://example.com/product/123",
    "extracted": {
      "title": "Wireless Headphones Pro",
      "price": 149.99,
      "in_stock": true
    }
  }
}

Response Fields

FieldTypeDescription
okbooleanWhether the request succeeded
data.urlstringThe URL you sent
data.extractedobject | array | stringWith schema: an object with your fields. With only prompt: JSON when the answer is structured, otherwise a string

Caching

Results are cached for 6 hours per URL, prompt, and schema.

Errors

StatusCodeWhen
400VALIDATION_ERRORurl is missing or invalid; neither prompt nor a usable schema was sent; prompt is over 2000 characters; or the URL is a Google page
404NOT_FOUNDThe target page returned 404 Not Found
429RATE_LIMITED_UPSTREAMToo many requests in flight — retry in a few seconds
502UPSTREAM_ERRORThe page could not be loaded or no answer came back — retry
503UPSTREAM_TIMEOUTThe page took too long to load — retry

Failed calls are not charged.

On this page