scantir api
two endpoints: fetch a url, or search google. json in, json out. you are billed only when it works.
authentication
every request carries your api key as a bearer token: Authorization: Bearer <key>. create and revoke keys in the console under keys. the key is shown once, at creation.
- no token, or a token scantir does not know →
401with{"error": "missing bearer token"}or{"error": "invalid api key"}. - the key service is unreachable →
502auth backend unavailable. nothing was fetched or billed; retry. - verified keys are cached for about 30 seconds, so a key you revoke can answer for up to 30 more seconds.
# every call, both endpoints curl https://api.scantir.com/v1/scrape \ -H "Authorization: Bearer $SCANTIR_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com" }'
{ "error": "invalid api key" }scrape
one url in, its content out. every scrape starts on the http tier: a tls-fingerprinted client (a real chrome or firefox profile with matching headers) through the managed residential pool, retrying on a fresh exit ip and fingerprint when the site walls it — up to six attempts, with a warmed-up retry (homepage first, then the target on the same cookie jar) when a cold request trips a soft gate. if every attempt is still blocked and you sent auto_escalate, the same url is rendered in a real browser on the best-ranked proxy; render: true skips the http tier entirely.
dns, proxy_tunnel, timeout…) still returns http 200, with error and error_type set. check error_type, not just the status code. phase 1 maps these to 502 / 503 / 504 with the same error_type. this page will say so when it lands.request body
| field | type | description | |
|---|---|---|---|
| url | string | required | absolute http(s) url. anything else is a 400. urls that point at private, internal or unresolvable addresses are refused with 400 egress denied. |
| render | boolean | skip the http tier and render in a real browser. default false. needed for screenshot and wait_selector. | |
| auto_escalate | boolean | if every http attempt is walled, retry the url in the browser tier. default false. the response then has escalated: true. | |
| screenshot | boolean | browser tier only. viewport screenshot, returned as screenshot_b64. default false. | |
| wait_selector | string | browser tier only. css selector to wait for before capturing, up to the timeout. an unmatched selector does not fail the request. | |
| extract | object | map of name → css selector. the response carries data (trimmed text of the first match per name, "" when nothing matches) instead of html. plain css only: no ::text / ::attr. | |
| country | string | two-letter iso code, lower-case (de). pins the proxy exit country for this request. without it scantir uses the country it has learned for the domain, or the whole pool. echoed back as country. | |
| timeout_ms | integer | per-attempt timeout: each http attempt and the browser render. 1,000–60,000 ms, default 30,000. a browser render is also capped by your plan: 15,000 ms on tinker, 30,000 on indie, 60,000 on team. |
response
always a ScrapeResult. optional fields are omitted when empty.
| url | string | the url fetched. the browser tier reports the final url after redirects. | |
| status | integer | upstream http status. 0 when nothing was fetched. | |
| method | string | tier that answered: http or browser. | |
| fingerprint | string | identity family the http tier used: chrome or firefox. absent on the browser tier. | |
| country | string | exit country pinned for this request. absent when the whole pool was used. | |
| waf | string | anti-bot detected in front of the response: cloudflare, datadome, perimeterx, akamai, imperva or none. | |
| blocked | boolean | the response is a challenge or block page, not content. also true for a 401, 403, 407, 429 or 503 from the target, and for every browser-tier failure. | |
| escalated | boolean | the http tier was walled and the browser tier answered. | |
| elapsed_ms | integer | wall time of the whole request, every attempt included. | |
| error_type | string | failure class when the fetch itself failed: bad_url, client, proxy_tunnel, pool_unavailable, dns, timeout, cancelled, tls, network, http_error (http tier); browser_error, queue, timeout, cancelled (browser tier). absent on success. | |
| truncated | boolean | true when the body was cut at the size cap: 10 MB on the http tier, 20 MB on the browser tier. absent otherwise. | |
| error | string | human-readable failure. absent on success. | |
| headers | object | upstream response headers, name → [values]. | |
| content_type | string | upstream content-type. | |
| body_type | string | html, json or text, from the content-type (json is also sniffed). | |
| html | string | the page, when body_type is html and no extract was sent. | |
| body | string | the body when it is not html (json, text). | |
| data | object | your extract names → text. replaces html. | |
| screenshot_b64 | string | base64 image, when screenshot was set and the browser tier ran. |
what is billed
a scrape counts toward your monthly quota only when it succeeded: no error, not blocked, and a 2xx or 3xx status. a 404 from the target, a wall, or a transport failure costs nothing. at the quota every call is 402 until the month resets (the 1st, utc).
# http first, browser if walled; two fields instead of html curl https://api.scantir.com/v1/scrape \ -H "Authorization: Bearer $SCANTIR_KEY" \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com", "auto_escalate": true, "extract": { "title": "h1", "price": ".price" } }'
{
"url": "https://example.com/",
"status": 200,
"method": "http",
"fingerprint": "chrome",
"waf": "none",
"blocked": false,
"escalated": false,
"elapsed_ms": 812,
"headers": {
"Content-Type": [
"text/html; charset=UTF-8"
]
},
"content_type": "text/html; charset=UTF-8",
"body_type": "html",
"data": { "title": "Example Domain", "price": "" }
}{
"url": "https://example.com/",
"status": 0,
"method": "browser",
"waf": "none",
"blocked": true,
"escalated": true,
"elapsed_ms": 40213,
"error_type": "timeout",
"error": "browser timeout waiting for worker"
}google serp
a query in, the results page as json out: organic results, ads, featured snippet, people also ask, related searches. results are localised with google's own gl / hl for the country and language you ask for, and every organic link is the real destination, resolved from google's redirect links before the page is returned.
request body
| field | type | description | |
|---|---|---|---|
| q | string | required | the search. max 512 characters. alias query. |
| country | string | two-letter code, default us. sets google's gl, the market results are ranked for. uk is accepted as gb. alias gl. | |
| language | string | google's hl, default en. regional forms like pt-BR. alias hl. | |
| location | string | google canonical location name, e.g. Austin,Texas,United States. max 200 characters; sent as uule. | |
| page | integer | 1–10, default 1. ten results a page. | |
| safe | boolean | safesearch on. default false. | |
| time_period | string | hour, day, week, month or year. | |
| no_cache | boolean | skip the cache and fetch live. the fresh page still refreshes it. | |
| include_html | boolean | also return the raw page as html. these responses are never cached. |
response
lists are always arrays, never null. featured_snippet and html are omitted when absent.
| search_parameters | object | the normalised query: q, country, language, location, page, safe, time_period, google_url. | |
| search_information | object | total_results, time_taken_seconds, query_displayed, spelling_fix ("showing results for…"), no_results. | |
| featured_snippet | object | title, link, displayed_link, snippet, list. | |
| ads | array | position, block (top | bottom), title, link, displayed_link, description. | |
| organic_results | array | position (1-based on this page), title, link, displayed_link, source, snippet, date. | |
| people_also_ask | array | question. | |
| related_searches | array | query, link. | |
| html | string | the raw page, only with include_html. | |
| meta | object | cached, method (http, browser, or google-serp — the dedicated search fleet), escalated, exit_country, elapsed_ms, credits (what this call cost: 1 for a fresh page, 0 from cache). |
caching and billing
- identical queries (same
q,country,language,location,page,safe,time_period) share one results page for one hour. a cache hit costs 0 credits and does not touch your hourly cap. - a fresh results page costs 1 credit. a page with no results is still a results page and is billed.
- every failure is free. a failed live fetch still uses one of the hour's slots, except a
503(no capacity), which gives it back. - serp has its own allowance, separate from your scrape quota: credits per month and live searches per hour, both set by your plan. see rate limits & quotas.
migrating from serpapi
the body accepts query, gl and hl as aliases for q, country and language, so a serpapi-shaped request works unchanged: {"query": "web scraping api", "gl": "us", "hl": "en"}. what differs: requests are POST with a json body, the key goes in the Authorization header instead of api_key, there is no num (ten results a page, use page), and location takes the same canonical names. organic_results keep the familiar position / title / link / snippet keys.
# page one, us results in english curl https://api.scantir.com/v1/serp/google \ -H "Authorization: Bearer $SCANTIR_KEY" \ -H "Content-Type: application/json" \ -d '{ "q": "web scraping api", "country": "us", "language": "en" }'
{
"search_parameters": {
"q": "web scraping api",
"country": "us",
"language": "en",
"page": 1,
"safe": false,
"google_url": "https://www.google.com/search?q=web+scraping+api&hl=en&gl=us&ie=UTF-8&pws=0"
},
"search_information": {
"total_results": 128000000,
"query_displayed": "web scraping api"
},
"ads": [],
"organic_results": [
{
"position": 1,
"title": "Web Scraping API — docs",
"link": "https://vendor.example/docs/api",
"displayed_link": "vendor.example › docs › api",
"source": "vendor.example",
"snippet": "Fetch any page as JSON with one POST request …"
}
],
"people_also_ask": [
{ "question": "Is web scraping legal?" }
],
"related_searches": [
{
"query": "web scraping api free",
"link": "https://www.google.com/search?q=web+scraping+api+free"
}
],
"meta": {
"cached": false,
"method": "google-serp",
"elapsed_ms": 6420,
"credits": 1
}
}# Retry-After: 1312 { "error": "hourly SERP limit reached (5/hour on your plan)", "error_type": "rate_limited" }
rate limits & quotas
scrapes count toward a monthly request quota, and only when they succeed, and your plan sets how many can run at once. google serp runs on shared capacity, so it has its own allowance: credits per month and live searches per hour. both are set by your plan and reset on the 1st (quota) and on the hour (hourly cap), utc.
| plan | requests / month | concurrent requests | serp credits / month | live serp searches / hour |
|---|---|---|---|---|
| tinker | 1,000 | 2 | 50 | 2 |
| indie | 100,000 | 20 | 1,000 | 5 |
| team | 1,000,000 | 100 | 5,000 | 10 |
| fleet | custom | custom | custom | custom |
- over the monthly request quota: every
/v1/scrapecall is402monthly quota exceeded. - over your concurrent requests:
/v1/scrapeis429rate_limitedwithRetry-After: 2. a slot frees as soon as one of your scrapes finishes; refused calls are not billed. - browser renders (
render, or anauto_escalateescalation) share a small pool, so each team runs at most 1 at a time on tinker and 2 on paid plans. over that, the same429. - over the monthly serp credits:
402quota_exceeded. - over the hourly serp cap:
429rate_limitedwith aRetry-Afterheader, seconds until the next utc hour. cache hits are not limited. - there is no per-second limit on
/v1/scrape. one url per call, synchronous; a call never runs longer thantimeout_msplus 15 seconds, so a client timeout of 90 seconds is enough.
{ "error": "monthly quota exceeded" }{
"error": "monthly SERP quota exceeded (50/month on your plan)",
"error_type": "quota_exceeded"
}errors
scrape errors are {"error": "message"}. serp errors, and the scrape 429, add error_type: {"error": "message", "error_type": "…"}. nothing that errors is billed.
| status | error_type | endpoint | meaning |
|---|---|---|---|
| 400 | — | scrape | invalid json body, url must be an absolute http(s) url, or egress denied: … (private, internal or unresolvable address). |
| 400 | invalid_request | serp | bad or missing parameter; error names it. |
| 401 | — | both | missing bearer token or invalid api key. |
| 402 | — | scrape | monthly quota exceeded. resets on the 1st, utc. |
| 402 | quota_exceeded | serp | this month's serp credits are spent. |
| 429 | rate_limited | scrape | concurrency limit reached (N at a time on your plan), or render limit reached (…) for browser renders. Retry-After: 2. |
| 429 | rate_limited | serp | hourly live-search cap reached. Retry-After = seconds to the next utc hour. |
| 502 | — | both | auth backend unavailable. retry. |
| 502 | blocked | serp | google walled every attempt (captcha, js wall, consent). retry shortly. |
| 502 | upstream | serp | transport failure, or result links that could not be resolved. retry shortly. |
| 502 | parse_failed | serp | page fetched but not parseable — most likely a google markup change. we watch this one. |
| 503 | unavailable | serp | no search capacity right now. Retry-After says when to try again. |
| 200 | various | scrape | the fetch itself failed: error_type is one of bad_url, client, proxy_tunnel, pool_unavailable, dns, timeout, cancelled, tls, network, http_error, browser_error, queue. today this is an http 200 ScrapeResult with error set — see what's changing. |
{
"error": "google served a captcha wall on every attempt; not billed, retry shortly",
"error_type": "blocked"
}