AI Scraping API
AI Scraping API — extract structured JSON from any webpage with a plain-English prompt or a typed schema. No CSS selectors. $0.03 per call.
Pricing, live playground and use cases: AI Scraping API on YepAPI
/v1/scrape/ai-extractThe AI Scraping API renders a page in a headless browser and has an AI model pull out the data you describe — no CSS selectors or XPath to write or maintain when the site's markup changes. Describe what you want with prompt, or pass a schema to get back an object with exactly the fields and types you defined.
For selector-based extraction at $0.01, use the Data Extraction API.
Usage
With a typed schema — data.extracted has exactly these fields:
const res = await fetch('https://api.yepapi.com/v1/scrape/ai-extract', {
method: 'POST',
headers: {
'x-api-key': 'YOUR_API_KEY',
'Content-Type': 'application/json',
},
body: JSON.stringify({
url: 'https://example.com/product/123',
schema: {
type: 'object',
properties: {
title: { type: 'string', description: 'Product name' },
price: { type: 'number', description: 'Current sale price' },
in_stock: { type: 'boolean', description: 'Whether it can be bought now' },
},
},
}),
});
const { data } = await res.json();
console.log(data.extracted); // { title, price, in_stock }curl -X POST https://api.yepapi.com/v1/scrape/ai-extract \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com/product/123", "schema": {"title": "the product name", "price": "price in USD", "in_stock": "whether it is in stock"}}'With only a prompt:
import requests
res = requests.post(
"https://api.yepapi.com/v1/scrape/ai-extract",
headers={"x-api-key": "YOUR_API_KEY"},
json={
"url": "https://example.com/product/123",
"prompt": "Return JSON with the product name, price, and a one-sentence description",
},
)
print(res.json()["data"]["extracted"])Request Body
| Parameter | Type | Required | Description | Default |
|---|---|---|---|---|
url | string | Yes | Full http(s) URL of the page (max 4096 characters) | — |
prompt | string | No | What to extract, in plain English (max 2000 characters). Required unless schema is given; with a schema it is used as extra context | — |
schema | object | No | Fields to extract: a JSON Schema with properties, or a flat {"field": "description"} map | — |
Prompt vs. Schema
- Prompt only:
data.extractedis the AI's answer. When the answer is structured — ask for JSON to get JSON — it is a JSON object or array; otherwise it is a string. - Schema:
data.extractedis an object with exactly the fields you listed. Two shapes are accepted:- JSON Schema:
{"type": "object", "properties": {"price": {"type": "number", "description": "Sale price"}, "title": {"type": "string"}}}. Supported types:string,number(integeris treated asnumber),boolean,array,object.descriptiontells the model what each field means; without it the field name is used. - Flat map:
{"title": "the product name", "price": "price in USD"}— each value is the field's description.
- JSON Schema:
Response
{
"ok": true,
"data": {
"url": "https://example.com/product/123",
"extracted": {
"title": "Wireless Headphones Pro",
"price": 149.99,
"in_stock": true
}
}
}Response Fields
| Field | Type | Description |
|---|---|---|
ok | boolean | Whether the request succeeded |
data.url | string | The URL you sent |
data.extracted | object | array | string | With schema: an object with your fields. With only prompt: JSON when the answer is structured, otherwise a string |
Caching
Results are cached for 6 hours per URL, prompt, and schema.
Errors
| Status | Code | When |
|---|---|---|
400 | VALIDATION_ERROR | url is missing or invalid; neither prompt nor a usable schema was sent; prompt is over 2000 characters; or the URL is a Google page |
404 | NOT_FOUND | The target page returned 404 Not Found |
429 | RATE_LIMITED_UPSTREAM | Too many requests in flight — retry in a few seconds |
502 | UPSTREAM_ERROR | The page could not be loaded or no answer came back — retry |
503 | UPSTREAM_TIMEOUT | The page took too long to load — retry |
Failed calls are not charged.
Related Endpoints
- Data Extraction API — CSS/XPath extraction rules at $0.01
- Web Scraping API — the whole page as Markdown for your own LLM
- Walmart Product API — ready-made structured data for Walmart product pages