> ## Documentation Index
> Fetch the complete documentation index at: https://docs.workfloo.ws/llms.txt
> Use this file to discover all available pages before exploring further.
# Scrape web API
Cost: **1 credit** (non-proxy)
Cost: **2 credits** (proxy)
Scrapes textual content from a given public website URL.
Allows optional proxying and multiple output formats: HTML, Markdown, or link extraction.
**Requires a valid API key.**
### Non-proxy scrape
When proxying is **not enabled,** scraping is performed using standard data center IPs. Some websites may block such traffic.
If proxy is **disabled,** the `response` includes:
```json theme={null}
"response": {
"success": true,
"proxy_used": false
}
```
### Proxy scrape
When proxying is **enabled,** the scraper follows these steps:
1. Attempts a regular (non-proxy) scrape.
2. If access is blocked, it retries using a proxy.
3. If a CAPTCHA is detected, it attempts to solve it.
4. The final result is returned to the client.
If proxy is **enabled,** the `response` includes additional diagnostic information:
```json theme={null}
"response": {
"success": true,
"proxy_used": true,
"bot_detected": true,
"captcha_detected": true,
"captcha_solved": true
}
```
## OpenAPI
````yaml api-reference/openapi-0.yaml post /scrape/web
openapi: 3.1.0
info:
title: Workfloows API
version: '0'
description: >
## Introduction
This API provides a growing collection of powerful tools and endpoints
commonly used in automation workflows. It's designed for individual
automators,
agencies, and app builders who want to streamline processes and connect
systems.
## Authentication
All requests require an API key, which must be included in the `x-api-key`
header.
```
x-api-key:
```
If the API key is missing or invalid, a `401 Unauthorized` response will be
returned.
servers:
- url: https://api.workfloo.ws/v0
security: []
paths:
/scrape/web:
post:
tags:
- Scrape
operationId: scrapeWeb
requestBody:
required: true
content:
application/json:
schema:
type: object
required:
- url
properties:
url:
type: string
format: uri
description: Target URL to scrape (including protocol)
example: https://workfloows.com
proxy:
type: object
description: Proxy settings
properties:
enabled:
type: boolean
default: false
description: Whether to enable proxy
output:
type: object
description: Output formatting options
properties:
html:
type: boolean
default: false
description: Return raw HTML
markdown:
type: boolean
default: false
description: Return Markdown format
links:
type: boolean
default: false
description: Include extracted links
responses:
'200':
description: Successful response
content:
application/json:
schema:
type: object
properties:
response:
type: object
properties:
success:
type: boolean
description: Whether the scraping was successful
proxy_used:
type: boolean
description: Whether a proxy was used for scraping
bot_detected:
type: boolean
description: Whether bot detection was triggered
captcha_detected:
type: boolean
description: Whether a CAPTCHA was detected
captcha_solved:
type: boolean
description: Whether a detected CAPTCHA was solved
results:
type: object
properties:
text:
type: string
description: The scraped content
html:
type: string
description: The raw HTML content
markdown:
type: string
description: The Markdown content
links:
type: array
description: The extracted links
items:
type: string
description: A link from the scraped page
examples:
regularSuccess:
summary: Success (no proxy used)
value:
response:
success: true
proxy_used: false
results:
text: Page content...
proxySuccess:
summary: Success (proxy used and CAPTCHA solved)
value:
response:
success: true
proxy_used: true
bot_detected: true
captcha_detected: true
captcha_solved: true
results:
text: Content of page behind the CAPTCHA...
'400':
description: Scraping error due to CAPTCHA or other failure
content:
application/json:
schema:
type: object
properties:
error:
type: string
description: Error message describing why the scraping failed
examples:
captchaFailed:
summary: Failed CAPTCHA solve
value:
error: >-
Detected CAPTCHA could not be solved. Scraping failed.
Credits refunded.
scrapeFailed:
summary: Scrape failed for other reasons
value:
error: Scraping failed. Credits refunded.
'401':
$ref: '#/components/responses/Unauthorized'
'403':
$ref: '#/components/responses/Forbidden'
'404':
$ref: '#/components/responses/NotFound'
'500':
$ref: '#/components/responses/InternalServerError'
security:
- apiKeyAuth: []
components:
responses:
Unauthorized:
description: Unauthorized
content:
application/json:
schema:
type: object
properties:
error:
type: string
examples:
unauthorized:
summary: Unauthorized
value:
error: Unauthorized.
apiKeyRequired:
summary: API Key required
value:
error: Unauthorized. API key is required.
Forbidden:
description: Forbidden
content:
application/json:
schema:
type: object
properties:
error:
type: string
examples:
invalidApiKey:
summary: Invalid API key
value:
error: Invalid API key.
apiKeyNotFound:
summary: API Key not found
value:
error: API Key not found.
rateLimitExceeded:
summary: Rate limit exceeded
value:
error: Rate limit exceeded.
NotFound:
description: Not Found
content:
application/json:
schema:
type: object
properties:
error:
type: string
examples:
userNotFound:
summary: User not found
value:
error: User not found.
InternalServerError:
description: Internal Server Error
content:
application/json:
schema:
type: object
properties:
error:
type: string
example: An error occurred while processing your request.
securitySchemes:
apiKeyAuth:
type: apiKey
in: header
name: x-api-key
````