POST
/
scraper
/
scrape
curl --request POST \
  --url https://api.usescraper.com/scraper/scrape \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "<string>",
  "format": "text",
  "extract_object": {},
  "advanced_proxy": false
}'
{
  "text": "Webpage content will be here...",
  "meta": {
    "url": "https://usescraper.com",
    "fetchedUrl": "https://usescraper.com",
    "fetchedUrlStatusCode": 200,
    "meta": {
      "title": "UseScraper - Web Crawler & Scraper API",
      "description": {}
    }
  },
  "status": "scraped"
}

Scrape any webpage URL and get text, html or markdown. A Chrome browser with Javascript enabled is used to ensure every webpage can be scraped. Set advanced_proxy to true to enable our advanced scraping proxy which will cicumvent most bot detection and blocking systems (using advanced proxy does not currently cost any extra).

{
  "text": "Webpage content will be here...",
  "meta": {
    "url": "https://usescraper.com",
    "fetchedUrl": "https://usescraper.com",
    "fetchedUrlStatusCode": 200,
    "meta": {
      "title": "UseScraper - Web Crawler & Scraper API",
      "description": {}
    }
  },
  "status": "scraped"
}

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json
URL to scrape & scraping options
​
url
string
required

URL to scrape

​
format
enum<string>
default:
text

Format to save all crawled page content to

Available options:
text,
html,
markdown
​
extract_object
object
​
advanced_proxy
boolean
default:
false

Use advanced proxy -- default is false

Response

200
application/json
Scraped content
​
status
enum<string>
required

Scraping status

Available options:
scraped,
failed
​
meta
object
required

Page metadata

​
text
string

Scraped content

WebhooksQuickstart