Lesson 02
Sending GET requests
A GET request through ScraperAPI is a single call: pass your API key and a target URL, and you get the fetched page back as the response body.
The pattern
Point any HTTP client at https://api.scraperapi.com/ with two query parameters and read the response body. That body is the page ScraperAPI fetched on your behalf.
curl "https://api.scraperapi.com/?api_key=YOUR_API_KEY&url=https://httpbin.org/anything"import requests
payload = {
"api_key": "YOUR_API_KEY",
"url": "https://httpbin.org/anything",
}
r = requests.get("https://api.scraperapi.com/", params=payload, timeout=70)
print(r.status_code)
print(r.text)import axios from "axios";
const { data, status } = await axios.get("https://api.scraperapi.com/", {
params: {
api_key: "YOUR_API_KEY",
url: "https://httpbin.org/anything",
},
timeout: 70000,
});
console.log(status);
console.log(data);<?php
$query = http_build_query([
"api_key" => "YOUR_API_KEY",
"url" => "https://httpbin.org/anything",
]);
$response = file_get_contents("https://api.scraperapi.com/?" . $query);
echo $response;require "net/http"
require "uri"
uri = URI("https://api.scraperapi.com/")
uri.query = URI.encode_www_form(api_key: "YOUR_API_KEY", url: "https://httpbin.org/anything")
puts Net::HTTP.get(uri)import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
String url = "https://api.scraperapi.com/?api_key=YOUR_API_KEY&url=https://httpbin.org/anything";
HttpClient client = HttpClient.newHttpClient();
HttpRequest req = HttpRequest.newBuilder(URI.create(url)).GET().build();
HttpResponse<String> res = client.send(req, HttpResponse.BodyHandlers.ofString());
System.out.println(res.body());package main
import (
"fmt"
"io"
"net/http"
"net/url"
)
func main() {
q := url.Values{}
q.Set("api_key", "YOUR_API_KEY")
q.Set("url", "https://httpbin.org/anything")
res, _ := http.Get("https://api.scraperapi.com/?" + q.Encode())
defer res.Body.Close()
body, _ := io.ReadAll(res.Body)
fmt.Println(string(body))
}Useful parameters
Beyond the two required parameters, a handful of optional ones cover most use cases:
| Name | Type | Description |
|---|---|---|
| render | boolean | Set to true to have ScraperAPI execute JavaScript on the target page before returning the HTML. |
| country_code | string | Two-letter code (us, gb, de) to route the request through a proxy in that country. |
| premium | boolean | Use ScraperAPI's premium proxy pool for the request. |
| keep_headers | boolean | Forward the custom request headers you send to ScraperAPI on to the target URL. |
Reading the response
The response body is exactly what the target server returned. The HTTP status code tells you how the request went overall:
200— request succeeded.401— your API key is missing or invalid.500— ScraperAPI could not complete the request; retry.
Tip
Set a generous client-side timeout (60–70 seconds is a good default). Fetching a page can involve retries on ScraperAPI's side, and cutting the request off too early loses those retries.