Scraper API is a proxy API for web scraping. They offer an API that handles proxies, browsers, and CAPTCHAs, so you can get the HTML from any web page with a simple API call. You have access to a pool of more than 20 million residential IPs. They offer unlimited bandwidth, only counting the number of API requests that you make.
Blazing Fast and Fully-Featured Proxy API
- Over 20 million residential IPs in the pool
- Simple dashboard to manage usage and billing
- Geo-targeting: target 12+ countries around the world
- Free plan with 1000 requests & all features
- Seven-day, no questions asked refund policy
- 24/7 support and great customer service
- Rotating and sticky IP sessions
- Easy setup
- Able to render JavaScript pages
- Custom browser headers
- Premium proxy pools
- Auto-extraction of data from popular sites
Scraper API provides a proxy service designed for web scraping. With over 20 million residential IPs across 12 countries, as well as software that can handle JavaScript rendering and solving CAPTCHAs, you can quickly complete large scraping jobs without ever having to worry about being blocked by any servers.
Implementation is extremely simple, and they offer unlimited bandwidth. Proxies are automatically rotated, but users can choose to maintain sessions if required. All you need to do is call the API with the URL that you want to scrape, and it will return the raw HTML. With Scraper API, you just focus on parsing the data, and they’ll handle the rest.
Scraper API Overview
Scraper API’s service is great for crawling large sites, especially e-commerce sites such as Amazon and eBay, and business directories. It can easily grab results from search engines. They use a residential IP network, which means their IPs are hard to detect and block, and their software is able to solve CAPTCHAs if they do come up. You won’t even notice – just focus on processing the HTML that you receive!
There’s a free tier for testing with 1000 requests and all the features. After that, there’s a Hobby tier ($29), a Startup tier ($99), and a Business tier ($249). Each tier allows for a different number of API calls, ranging from 250,000 per month for the Hobby tier to 3,000,000 requests per month for the Business tier.
If you need more, or have different requirements, just send the team a message and they will create a custom plan for you that suits your needs.
Is Scraper API the right tool for you? We’ll dive deeper into what they offer in this review, so you can decide.
Get 10% off for the first month discount: put promo code “SCRAPE38698” to check out!
The Scraper API solution
- Scraper API automatically rotates IP addresses from a pool of over 20 million residential IPs across a dozen ISPs, and will automatically retry failed requests. If the page exists, you will be able to access it.
- The advanced software on Scraper API will solve any CAPTCHAs that come up.
- It can be used to scrape pages that are rendered in Javascript – simply set ‘render=true’ in your API call, and it will render the page with a headless browser and return the data to you.
- You can set custom headers, so websites won’t be able to distinguish your activity from normal browsing activity.
- They have servers in 12+ different countries, and you can easily set which location you want your requests to come from.
- Unlimited bandwidth – unlike other proxy services, Scraper API will not charge you for bandwidth – you are charged based on the number of requests you require and the options you require.
- The servers are fast, with speeds up to 100Mb/s, and a guaranteed 99.9% uptime.
- They offer professional support, with fast turnaround times
How Scraper API works
When you sign up for Scraper API you are given an access key. All you need to do is call the API with your key and the URL that you want to scrape, and you will receive the raw HTML of the page as a result. It’s as simple as:
curl “http://api.scraperapi.com?api_key=3ff1f5e972791859ac9960827414be23&url=http://httpbin.org/ip”
On the back end, when Scraper API receives your request, their service accesses the URL via one of their proxy servers, gets the data, and then sends it back to you.
IP rotation is automated on their side, so you don’t need to do anything besides call the API unless you have specific needs, such as Javascript rendering, requiring IPs from a specific location, or extra headers. If you need those features, you just have to add some extra text at the end of the API call, but we’ll get into more detail about that later.
The plans all come with a certain amount of concurrent requests, so you can scrape things faster. If you send too many requests at the same time, Scraper API will return a 429 status code, which just means you’ll need to slow down your requests a little bit, but they won’t penalize you for it.
back to menu ↑Pricing packages of Scraper API
Scraper API has three standard pricing plans, a free plan, and you can also request custom plans.
There is a 7-day, no questions asked refund policy. The free plan doesn’t require you to enter any payment information, so it is totally commitment free.
The hobby plan is $29 per month and it gives you 250,000 API calls on their standard proxies with a maximum of 10 concurrent requests. On the Startup plan, you receive 1,000,000 API calls on their standard proxies with up to 25 concurrent requests. You also have the option to set your IPs to only come from US or DE servers. It costs $99 per month.
The Business plan is $249 per month, and it gives you 3,000,000 API calls per month. You will be able to choose from 12 countries for where your IPs originate from. This plan also allows Javascript rendering, premium proxies, and you are allowed up to 50 concurrent requests.
If you need more concurrent requests or API calls per month, you can contact the sales team to organize an Enterprise Custom plan.
Business and Enterprise customers receive priority email support.
back to menu ↑How to use Scraper API
Scraper API’s free plan will give you 1000 requests per month and will have all the features available, so you can test them out for yourself.
When you log into your Scraper API account, you will be presented with a dashboard that will show you how many requests you have used, how many requests you have left for the month, and the number of failed requests (which do not count towards your request limit).
The main page will also display your API key and also an example link that you can copy and paste straight into your code.
On the left sidebar, you will see a link to their Documentation page, which has clear instructions for each of their features. It contains code that can be copied and pasted straight into your scripts for Bash, Node, Python/Scrapy, PHP, and Ruby.
By default, your IPs will be automatically rotated for every request – CAPTCHA are automatically solved on the server side.
Basic Usage
The basic method of accessing the API is:
http://api.scraperapi.com/?api_key=[yourAPIkey]&url=[URL]
You can even try accessing the key via a browser. The server will then return the HTML for the webpage, which you can then store as a variable:
<html> <head> </head> <body> Hello world </body> </html>
It’s really that simple.
Rendering Javascript
If you need to render Javascript, all you need to do is add ‘&render=true’ to the end of your API call, for example:
http://api.scraperapi.com/?api_key=[yourAPIkey]&url=[URL]&render=true
The Scraper API servers will use a headless Chrome instance to visit the page, render it, and return the code back to you.
Custom Headers
If you need to keep your original request headers to keep the same user agents, cookies, etc, just add ‘&keep_headers=true’ to the results. This is only for getting customized results, rather than avoiding blocks, as Scraper API’s software will handle blocks internally.
http://api.scraperapi.com/?api_key=[yourAPIkey]&url=[URL]&keep_headers=true
Sessions
You can keep using the same proxy for multiple usages. Sessions will expire 60 seconds after the last usage. All you need to do is add ‘&session_number=[number]’ to your request – the number can be any integer. For example:
http://api.scraperapi.com/?api_key=[yourAPIkey]&url=[URL]&session_number=123
Geographic Location
Proxies can be set to come from specific locations using ‘&country_code=[flag]’. Users on the Startup plan and higher will be able to set proxy location to US. Business plan customers will have access to proxies in Canada (code=ca), United Kingdom (code=uk), Germany (code=de), France (code=fr), Spain (code=es), Brazil (code=br), Mexico (code=mx), India (code=in), Japan (code=jp), China (code=cn), and Australia (code=au). Example:
http://api.scraperapi.com/?api_key=[yourAPIkey]&url=[URL]&country_code=us
Premium Proxy Pools
Some sites can be harder to scrape, so Scraper API reserves a private internal pool of residential and mobile IPs that are available to users on Business or Enterprise plans. These IPs are charged at 10 times the rate of normal IPs (ie. they will count as 10 requests instead of one), and if Javascript rendering is required then each request will count as 25 requests. To use premium IPs, just add ‘&premium=true’ to your requests:
http://api.scraperapi.com/?api_key=[yourAPIkey]&url=[URL]&premium=true
Account Information
Account InformationYou can write a program to automate the monitoring of your account. Just send a request to the API without any URL or extra settings, and it will return your current account status:
http://api.scraperapi.com/?api_key=[yourAPIkey]
Example result: { "concurrentRequests":553, "requestCount":6655888, "failedRequestCount":1118, "requestLimit":10000000, "concurrencyLimit":1000 }back to menu ↑
Scraper API ISP Testing
For Scraper API, only offer proxy API to scrape the data, so it's hard to test their speed, So here we easily test some of their ISP.
So you know they’re using real residential IPs.
Our Verdict
Scraper API is the best proxy API service for web scraping on the market today. Their service is easy to integrate and use, and solve a range of problems for scraping bots, including Javascript rendering, proxy rotation, and CAPTCHA resolution.
They are able to accommodate for all levels/sizes of scraping projects. Scraper API has fast proxies in multiple countries, and you can specify where you want your requests to be sent from. If you have any serious scraping projects, then Scraper API is definitely worth looking into. Even if you’re a casual user, you may benefit from using the free plan.
Thanks for taking the time to read this review! If you have purchased a plan from Scraper API in the past and have a review you’d like to share, feel free to leave it in the comments below. We’d love to hear your opinion!
Get 10% off for the first month discount: put promo code “SCRAPE38698” to check out!
Scraper API Features
- Scraper API has a residential IP network of over 20 million addresses. No website will be able to detect where your requests are coming from. The chances of you using the same IP for a site twice are extremely slim, and since the addresses are residential, they look like they come from real people.
- Geo-targeting allows you to make sure your IP addresses originate from a specific country, so you won’t have any issues accessing websites that block or restrict certain areas.
- A professional customer support team is available 24/7 via email and live chat to help you.
- Scraper API’s proxies can provide speeds of up to 100 Mb/s, and they have a 99.9% uptime guarantee.
- They have a 7-day, no questions asked money back guarantee, so you can use their service without any worries.
- You can set your scrapers to high speed with concurrent connections.
- Pages that render in Javascript will no longer be an issue because Scraper API will render the pages for you.
- Never encounter any CAPTCHAs again – Scraper API will solve any captchas for you.
- Set custom headers for websites that block certain browsers or display different information depending on the browser.
- Maintain sessions easily – just change a setting, and keep your IP
- Monitor and track your usage with Scraper API’s simple and easy to use dashboard.
- All you need to do is set your scripts to call the API and then parse the data you receive – it’s simple and easy to use.
- Autoextract pages into useful data on popular sites like Amazon, Google, Instagram, and more.
[/wpsm_box]