GET
/
crawler
/
jobs
/
{id}
/
data
curl --request GET \
  --url https://api.usescraper.com/crawler/jobs/{id}/data \
  --header 'Authorization: Bearer <token>'
{
    "data": [
        {
            "meta": {
                "page": 1,
                "url": "https://usescraper.com/blog",
                "fetchedUrl": "https://usescraper.com/blog",
                "meta": {
                    "title": "Blog - UseScraper",
                    "description": "UseScraper Blog - Hyper fast web crawling and scraping API. Browser rendering, Markdown output and more."
                }
            },
            "text": "# Blog ...rest of page content..."
        },
        // ...more results
  ]
}

Use this endpoint to fetch the results of a successful crawler job. The results are returned as an array of objects, each containing meta (info about the crawled page) and text (page content in the format requested).

The maximum results per request in 1,000. If there are more results to fetch, a cursor will be returned in the response. Include this cursor as a URL query param in the next request to fetch more results.

{
    "data": [
        {
            "meta": {
                "page": 1,
                "url": "https://usescraper.com/blog",
                "fetchedUrl": "https://usescraper.com/blog",
                "meta": {
                    "title": "Blog - UseScraper",
                    "description": "UseScraper Blog - Hyper fast web crawling and scraping API. Browser rendering, Markdown output and more."
                }
            },
            "text": "# Blog ...rest of page content..."
        },
        // ...more results
  ]
}

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

​
id
string
required

Job ID

Query Parameters

​
limit
number | null
default:400

Number of results to return per request (max 500 which is also the default)

Required range: x <= 400
​
cursor
string

Cursor for pagination, returned in response

Response

200 - application/json

Crawled pages content and metadata

The response is of type object.