Web Scraping
Standard Scrape
Scrape any URL and get back clean markdown, HTML, or structured data.
POST
$0.01/call/v1/scrapeUsage
const res = await fetch('https://api.yepapi.com/v1/scrape', {
method: 'POST',
headers: {
'x-api-key': 'YOUR_API_KEY',
'Content-Type': 'application/json',
},
body: JSON.stringify({ url: 'https://example.com', format: 'markdown' }),
});
const { data } = await res.json();
console.log(data.markdown);curl -X POST https://api.yepapi.com/v1/scrape \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "format": "markdown"}'Request Body
| Parameter | Type | Required | Description | Default |
|---|---|---|---|---|
url | string | Yes | URL to scrape | — |
format | string | No | Output format: markdown, html, text | "markdown" |
waitFor | number | No | Wait for JS rendering (ms) | 0 |
selector | string | No | CSS selector to extract specific element | — |
Response
{
"ok": true,
"data": {
"url": "https://example.com",
"statusCode": 200,
"title": "Example Domain",
"markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
"metadata": {
"description": "Example Domain",
"ogTitle": "Example Domain"
}
}
}Response Fields
| Field | Type | Description |
|---|---|---|
ok | boolean | Whether the request succeeded |
data | object | Response payload |
data.url | string | The URL that was scraped |
data.statusCode | number | HTTP status code returned by the target page |
data.title | string | Page title extracted from the <title> tag |
data.markdown | string | Page content converted to clean Markdown |
data.metadata | object | Extracted page metadata |
data.metadata.description | string | Meta description of the page |
data.metadata.ogTitle | string | Open Graph title, if present |
Under the Hood
Under the hood, this uses our scraping infrastructure with JS rendering enabled by default. We handle proxy rotation, CAPTCHA solving, and convert the raw HTML to clean markdown automatically.