ScrapingBee Logo

The ScrapingBee API is one of the smart proxy APIs in the market that you can use to evade anti-bot and anti-scraping systems. According to the information on its website – it is the tool for you if you are tired of getting block as it helps you handle proxies and headless browsers. Aside from these, it does more, and you will learn about this as we progress in the review.

Advanced and Feature-rich Proxy API

  • Support for JavaScript rendering
  • Large proxy pool size and support for geo-location targeting
  • Smart IP rotation system
  • You only get to pay for successful requests
  • Dedicated account manager for the business and enterprise plan
  • Support many modes – proxy, HTML API, and Google search
  • Easy to understand documentation
  • Free trial available


The ScrapingBee service is a proxy API service that has been designed to help automation experts evade anti-bot systems by smartly rotating your IP address, solving Captcha, and hiding footprints that would reveal your bots’ actions.

This service is basically a proxy service under the hood, but it does more than regular proxies. In fact, except you use the proxy mode support, you would not even see it as a proxy service as you can use it as a simple HTTP API. With this service, you will not have to think of blocks as it takes care of that for you. Aside from helping you to avoid blocks, this service also has support for JavaScript rendering using the latest version of headless Chrome.

ScrapingBee overview

The service has got a large proxy pool with IPs from many countries across the globe. One thing you will come to like about this service is that you are only charged when your request is successful. This means that for them to make money, they need to be effective and aid your project.

Our research reveals to us that the ScrapingBee service is one of the best proxy API services in the market. And we are not alone; there are a good number of popular businesses that make use of the ScrapingBee service for their data collection projects, some of which include Zillow, Contently, Sap, Zapier, Deloitte, and Woo Commerce, among others.

Is the service worth its salt? That’s what we’ll discuss in this review.


Expert Review & Guides

[wpsm_toplist]

ScrapingBee features


The ScrapingBee Proxy Solution

The ScrapingBee service was developed to help web scrapers access web content without hindrance and blocks. Aside from this,you can use the service as a general proxy service by using the ScrapingBee proxy mode. Let take a look at some of the services provided by this company.

API Mode:

The API mode is the main proposition of this service. For this service, all you need to do is send a simple API request, and you get the content of the page you want – no setting proxies, no handling of JS on your part – just a simple API request. For this mode, they have got support for regular HTTP API. They also provide libraries and clients to use for popular programming languages, including Python, NodeJS, Java, Ruby, PHP, and Go.

Proxy Mode:

The API mode above has been developed for web scraping. If you need to use their service for other tasks, you can use their proxy mode,which would allow you to use the service as a regular proxy service. You are provided a hostname/proxy address, proxy port, and username and password for authentication.

Google Search and other Web Service API:

Do you want to scrap data from the Google SERPs? Then you do not even need to develop your own scraper as ScrapingBee provides you a Google Scraper in the form of an API or as a client to be used in your codes – Python, NodeJS, Java, PHP, and Go. Aside from Google, ScrapingBee has support for Booking, Google Maps, and TripAdvisor.

Data Extraction API:

This offer is only available for Python, NodeJS, Ruby, PHP, and Go developers. This tool is purely a library for extracting data from HTML pages using CSS selectors. With this, you might not even need another parsing tool, except you are looking for something advanced.

20% OFF
20% OFF

Get 20% Life-time discount: put promo code “privateproxyreviews” to check out!


How ScrapingBee Works

As stated earlier, the ScrapingBee service is an advanced proxy system that does more than regular proxy services. It has been developed to be easy to use, and if you are a proxy user, then you can find their service even easier to use as there is no tedious setup required except you want to make use of their advanced features. The working mechanism is simple and works the same way regular proxies do. It sits in the middle between your device and web servers on the Internet. However, unlike regular proxies that reroute your request, the ScrapingBee service does more.

When ScrapingBee gets your request, it look at the URL and get the URL of the page and the associated data, and using that; it helps you get the content of the page, sending it via proxy servers, and rotating your IP as per your requirement. If a request is successful, it sends you the response and deducts the appropriate credit from your credit balance. However, if the request wasn’t successful, it would retry, after which if it gets the same response, you will be notified, and nothing would be deducted from your credit balance.


Pricing & Features

Pricing is generally one of the most important factors that influence the consumers buying decisions, and unfortunately, it is not decided just like that, and a lot of factors come into play which usually makes premium services more expensive. How would we rate the pricing of ScrapingBee?

The ScrapingBee service can be said to be competitive, consumer-focused, and result-oriented, as you only get to pay for successful requests. In fact, to prove that new users are offered 1000 free API credits to test out the service and see if the service works for their use case before making a monetary commitment.

ScrapingBee Pricing

In terms of packages, the service has 3 packages publicly displayed on their website. The smallest package is the startup package that comes with 1000,000 API credits and is sold for $99. This package has a concurrent limit of 10. The have got the business package with 2,500,000 API credit sold at a price of $249.

If you want a bigger plan, you will need to contact them for a quote, and you will get a discount. The pricing system is not all good. In fact, if you plan to use advanced features such as JavaScript rendering, and their premium proxies, then that costs you additional API credits. For a better pricing system, you could take a look at the offers of Scraper API as this service even offers a hobbyist package that is sold for $29.

Features of ScrapingBee

  • Rotating proxies that change your IP address in a smart way to avoid detection. If you want, they can use IPs from their premium pool, but that would mean spending more API credit.
  • It has support for geo-location targeting where you can choose to use proxies from selected countries.
  • It has support for rendering JavaScript. This makes it a more advanced proxy API service when compared to other proxy API such as the Zyte Smart Proxy Manager that requires you to use a browser automator as an add-on. This service uses headless Chrome to render JavaScript.

Features of ScrapingBee

  • Has support for libraries and clients for popular programming languages including Python, NodeJS, Java, Ruby, PHP, and Go. For these, they have got scrapers available for some web services, including Google Map and Search, Booking, and TripAdvisor.

ScrapingBee support for libraries

  • You only get charged for successful requests, and as such, for the service to earn money, your request needs to be successful, but you will have to buy API credit in advanced – deduction is after every successful request.
  • They offer a free 1000 API credit plan for new users to test out the service.
  • Developed specifically for developers by developers, and as such, they have got extensive documentation that provides you guidance on how to best use their service.
  • Provide support for not only API mode but also regular proxy mode so you could use their service to evade ban in your custom automation bot.

How to Use ScrapingBee?

The ScrapingBee tools are some of the easiest to use in the market, provided you can dig into their documentation. Hopefully, this shouldn’t be a problem for you as a web scraper. While the tool can be said to be simple to use, the ease of usage depends largely on the tool and the features you want to make use of. Let take a look at a brief guide on how to use each of the tools being discussed in this article.

How to use ScrapingBee API Mode

This is the easiest to use of the tools supported. One thing you will come to like is that you can send simple API calls as you send web requests. Also, Scraping API provides libraries for supported language so that it is made easy to integrate into your code. Let take a look at how to use ScrapingBee in API mode.

curl"https://app.scrapingbee.com/api/v1/?api_key=YOUR-API-KEY&url=YOUR-URL"

Looking at the code above, you can see that that the base URL is https://app.scrapingbee.com/api/ and then the version of the API, your API KEY, and the URL of the page you want to scrap. The same code in the Python version is provided below.

# Install the Python ScrapingBee library:
# pip install scrapingbee
fromscrapingbeeimportScrapingBeeClient
client = ScrapingBeeClient(api_key='YOUR-API-KEY')
response = client.get('YOUR-URL')
print('Response HTTP Status Code: ', response.status_code)
print('Response HTTP Response Body: ', response.content)

To learn more about how to use the ScrapingBee API mode, read the API mode documentation.

How to use ScrapingBee Proxy Mode

The proxy mode will allow you to use the service as a regular proxy with a proxy address, port, username, and password.

HTTP address: proxy.scrapingbee.com:8886
HTTPS address: proxy.scrapingbee.com:8887
Socks5 address: socks.scrapingbee.com:8888
Username: YOUR-API-KEY
Password: PARAMETERS

ScrapingBee provides a comprehensive guide on how to integrate the proxy mode with popular automation tools, including Apify, Kameleo, Octoparse, and Parsehub, among others. You can learn about this here.

How to Use Data Extraction Tool

From the above, you already know that ScrapingBee has a parsing tool you can use to extract data using CSS selectors. This is purely for programmers as you cannot use HTTP API requests to parse, and it is available for Python, NodeJS, Go, Ruby, and PHP. Let me show you a basic example of how to make use of the Data Extraction tool using the Python programming language.

# Install the Python ScrapingBee library:
# pip install scrapingbee
fromscrapingbeeimportScrapingBeeClient
client = ScrapingBeeClient(api_key='YOUR-API-KEY')
response = client.get(
'https://www.scrapingbee.com/blog',
params={
'extract_rules':{"title": "h1", "subtitle": "#subtitle"},
                        },
)
print('Response HTTP Status Code: ', response.status_code)
print('Response HTTP Response Body: ', response.content)

If you look at the code, you will see that no new library has been introduced – we used the same from scrapingbee import ScrapingBeeClient. To learn more about using CSS selectors to parse data from web pages, read the Data Extraction documentation.


Are ScrapingBee IPs Residential or Not?

The type of IP address you use will determine if you are easily detected and blocked while carrying out your web scraping tasks. Generally, the ScrapingBee makes use of both datacenter and residential IP addresses in a bid to provide you optimal performance at friendly pricing. It might interest you to know that not only ScrapingBee does this; most proxy APIs do this. However, if you ask me, this shouldn’t be a problem since you only have to pay for successful requests. If an IP is blocked, it is up to them, and IPs are rotated, so you do not have anything to worry about.


Our Verdict

If you ask me, I will tell you that the ScrapingBee service has proven itself to be a service you can trust. With this service, you do not need to think of blocks, and if a request fails, you are not charged for it.  Even though it is not the cheapest in the market, it has proven to be one of the best. The service is recommended and, in fact, one of our top 3 proxy API providers in the market. This does not mean it does not have downsides. However, the upsides of this service are way more than its downsides.

Update - 24.07.2021
9.5 Total Score
Trustable

Working Proxy API, Google Search and other Web Service API

Price & Value
9.5
Network Perform
9.3
Proxy Features
9.6
Customer Support
9.4
Add your review

ScrapingBee Logo

The ScrapingBee API is one of the smart proxy APIs in the market that you can use to evade anti-bot and anti-scraping systems. According to the information on its website – it is the tool for you if you are tired of getting block as it helps you handle proxies and headless browsers. Aside from these, it does more, and you will learn about this as we progress in the review.

Advanced and Feature-rich Proxy API

  • Support for JavaScript rendering
  • Large proxy pool size and support for geo-location targeting
  • Smart IP rotation system
  • You only get to pay for successful requests
  • Dedicated account manager for the business and enterprise plan
  • Support many modes – proxy, HTML API, and Google search
  • Easy to understand documentation
  • Free trial available


The ScrapingBee service is a proxy API service that has been designed to help automation experts evade anti-bot systems by smartly rotating your IP address, solving Captcha, and hiding footprints that would reveal your bots’ actions. This service is basically a proxy service under the hood, but it does more than regular proxies. In fact, except you use the proxy mode support, you would not even see it as a proxy service as you can use it as a simple HTTP API. With this service, you will not have to think of blocks as it takes care of that for you. Aside from helping you to avoid blocks, this service also has support for JavaScript rendering using the latest version of headless Chrome.

ScrapingBee overview

The service has got a large proxy pool with IPs from many countries across the globe. One thing you will come to like about this service is that you are only charged when your request is successful. This means that for them to make money, they need to be effective and aid your project.

Our research reveals to us that the ScrapingBee service is one of the best proxy API services in the market. And we are not alone; there are a good number of popular businesses that make use of the ScrapingBee service for their data collection projects, some of which include Zillow, Contently, Sap, Zapier, Deloitte, and Woo Commerce, among others.

Is the service worth its salt? That’s what we’ll discuss in this review.



The ScrapingBee Proxy Solution

The ScrapingBee service was developed to help web scrapers access web content without hindrance and blocks. Aside from this, you can use the service as a general proxy service by using the ScrapingBee proxy mode. Let take a look at some of the services provided by this company.

API Mode:

The API mode is the main proposition of this service. For this service, all you need to do is send a simple API request, and you get the content of the page you want – no setting proxies, no handling of JS on your part – just a simple API request. For this mode, they have got support for regular HTTP API. They also provide libraries and clients to use for popular programming languages, including Python, NodeJS, Java, Ruby, PHP, and Go.

Proxy Mode:

The API mode above has been developed for web scraping. If you need to use their service for other tasks, you can use their proxy mode,which would allow you to use the service as a regular proxy service. You are provided a hostname/proxy address, proxy port, and username and password for authentication.

Google Search and other Web Service API:

Do you want to scrap data from the Google SERPs? Then you do not even need to develop your own scraper as ScrapingBee provides you a Google Scraper in the form of an API or as a client to be used in your codes – Python, NodeJS, Java, PHP, and Go. Aside from Google, ScrapingBee has support for Booking, Google Maps, and TripAdvisor.

Data Extraction API:

This offer is only available for Python, NodeJS, Ruby, PHP, and Go developers. This tool is purely a library for extracting data from HTML pages using CSS selectors. With this, you might not even need another parsing tool, except you are looking for something advanced.

20% OFF
20% OFF

Get 20% Life-time discount: put promo code “privateproxyreviews” to check out!


back to menu ↑

How ScrapingBee Works

As stated earlier, the ScrapingBee service is an advanced proxy system that does more than regular proxy services. It has been developed to be easy to use, and if you are a proxy user, then you can find their service even easier to use as there is no tedious setup required except you want to make use of their advanced features. The working mechanism is simple and works the same way regular proxies do. It sits in the middle between your device and web servers on the Internet. However, unlike regular proxies that reroute your request, the ScrapingBee service does more.

When ScrapingBee gets your request, it look at the URL and get the URL of the page and the associated data, and using that; it helps you get the content of the page, sending it via proxy servers, and rotating your IP as per your requirement. If a request is successful, it sends you the response and deducts the appropriate credit from your credit balance. However, if the request wasn’t successful, it would retry, after which if it gets the same response, you will be notified, and nothing would be deducted from your credit balance.


back to menu ↑

Pricing & Features

Pricing is generally one of the most important factors that influence the consumers buying decisions, and unfortunately, it is not decided just like that, and a lot of factors come into play which usually makes premium services more expensive. How would we rate the pricing of ScrapingBee? The ScrapingBee service can be said to be competitive, consumer-focused, and result-oriented, as you only get to pay for successful requests. In fact, to prove that new users are offered 1000 free API credits to test out the service and see if the service works for their use case before making a monetary commitment.

ScrapingBee Pricing

In terms of packages, the service has 3 packages publicly displayed on their website. The smallest package is the startup package that comes with 1000,000 API credits and is sold for $99. This package has a concurrent limit of 10. The have got the business package with 2,500,000 API credit sold at a price of $249. If you want a bigger plan, you will need to contact them for a quote, and you will get a discount. The pricing system is not all good. In fact, if you plan to use advanced features such as JavaScript rendering, and their premium proxies, then that costs you additional API credits. For a better pricing system, you could take a look at the offers of Scraper API as this service even offers a hobbyist package that is sold for $29.

Features of ScrapingBee

  • Rotating proxies that change your IP address in a smart way to avoid detection. If you want, they can use IPs from their premium pool, but that would mean spending more API credit.
  • It has support for geo-location targeting where you can choose to use proxies from selected countries.
  • It has support for rendering JavaScript. This makes it a more advanced proxy API service when compared to other proxy API such as the Zyte Smart Proxy Manager that requires you to use a browser automator as an add-on. This service uses headless Chrome to render JavaScript.

Features of ScrapingBee

  • Has support for libraries and clients for popular programming languages including Python, NodeJS, Java, Ruby, PHP, and Go. For these, they have got scrapers available for some web services, including Google Map and Search, Booking, and TripAdvisor.

ScrapingBee support for libraries

  • You only get charged for successful requests, and as such, for the service to earn money, your request needs to be successful, but you will have to buy API credit in advanced – deduction is after every successful request.
  • They offer a free 1000 API credit plan for new users to test out the service.
  • Developed specifically for developers by developers, and as such, they have got extensive documentation that provides you guidance on how to best use their service.
  • Provide support for not only API mode but also regular proxy mode so you could use their service to evade ban in your custom automation bot.

back to menu ↑

How to Use ScrapingBee?

The ScrapingBee tools are some of the easiest to use in the market, provided you can dig into their documentation. Hopefully, this shouldn’t be a problem for you as a web scraper. While the tool can be said to be simple to use, the ease of usage depends largely on the tool and the features you want to make use of. Let take a look at a brief guide on how to use each of the tools being discussed in this article.

How to use ScrapingBee API Mode

This is the easiest to use of the tools supported. One thing you will come to like is that you can send simple API calls as you send web requests. Also, Scraping API provides libraries for supported language so that it is made easy to integrate into your code. Let take a look at how to use ScrapingBee in API mode.

curl"https://app.scrapingbee.com/api/v1/?api_key=YOUR-API-KEY&url=YOUR-URL"

Looking at the code above, you can see that that the base URL is https://app.scrapingbee.com/api/ and then the version of the API, your API KEY, and the URL of the page you want to scrap. The same code in the Python version is provided below.

# Install the Python ScrapingBee library:
# pip install scrapingbee
fromscrapingbeeimportScrapingBeeClient
client = ScrapingBeeClient(api_key='YOUR-API-KEY')
response = client.get('YOUR-URL')
print('Response HTTP Status Code: ', response.status_code)
print('Response HTTP Response Body: ', response.content)

To learn more about how to use the ScrapingBee API mode, read the API mode documentation.

How to use ScrapingBee Proxy Mode

The proxy mode will allow you to use the service as a regular proxy with a proxy address, port, username, and password.

HTTP address: proxy.scrapingbee.com:8886
HTTPS address: proxy.scrapingbee.com:8887
Socks5 address: socks.scrapingbee.com:8888
Username: YOUR-API-KEY
Password: PARAMETERS

ScrapingBee provides a comprehensive guide on how to integrate the proxy mode with popular automation tools, including Apify, Kameleo, Octoparse, and Parsehub, among others. You can learn about this here.

How to Use Data Extraction Tool

From the above, you already know that ScrapingBee has a parsing tool you can use to extract data using CSS selectors. This is purely for programmers as you cannot use HTTP API requests to parse, and it is available for Python, NodeJS, Go, Ruby, and PHP. Let me show you a basic example of how to make use of the Data Extraction tool using the Python programming language.

# Install the Python ScrapingBee library:
# pip install scrapingbee
fromscrapingbeeimportScrapingBeeClient
client = ScrapingBeeClient(api_key='YOUR-API-KEY')
response = client.get(
'https://www.scrapingbee.com/blog',
params={
'extract_rules':{"title": "h1", "subtitle": "#subtitle"},
                        },
)
print('Response HTTP Status Code: ', response.status_code)
print('Response HTTP Response Body: ', response.content)

If you look at the code, you will see that no new library has been introduced – we used the same from scrapingbee import ScrapingBeeClient. To learn more about using CSS selectors to parse data from web pages, read the Data Extraction documentation.


back to menu ↑

Are ScrapingBee IPs Residential or Not?

The type of IP address you use will determine if you are easily detected and blocked while carrying out your web scraping tasks. Generally, the ScrapingBee makes use of both datacenter and residential IP addresses in a bid to provide you optimal performance at friendly pricing.

It might interest you to know that not only ScrapingBee does this; most proxy APIs do this. However, if you ask me, this shouldn’t be a problem since you only have to pay for successful requests. If an IP is blocked, it is up to them, and IPs are rotated, so you do not have anything to worry about.


back to menu ↑

Our Verdict

If you ask me, I will tell you that the ScrapingBee service has proven itself to be a service you can trust. With this service, you do not need to think of blocks, and if a request fails, you are not charged for it.  Even though it is not the cheapest in the market, it has proven to be one of the best.

The service is recommended and, in fact, one of our top 3 proxy API providers in the market. This does not mean it does not have downsides. However, the upsides of this service are way more than its downsides.

Update - 24.07.2021
9.5 Total Score
Trustable

Working Proxy API, Google Search and other Web Service API

Price & Value
9.5
Network Perform
9.3
Proxy Features
9.6
Customer Support
9.4
Add your review

We will be happy to hear your thoughts

      Leave a reply

      Your total score

      Pin It on Pinterest

      Private Proxy Reviews