How to Setup Proxies on Octoparse
This section is for those that do not know how to set up proxies on Octoparse. If you know how to get that done, you can move over to the next section. From the list above, you should make a choice on the provider to buy proxies from.
Make a purchase and get the proxy credentials — proxy address, port, username, and password. Username and password are not a must if the provider supports IP authentication. Follow the steps below to set them up.
1. In order to set up proxies, you must create a workflow. Create one.
2. Go to the workflow you want to set up the proxies and click on “Settings”
3. Select “Use Proxies” and then “Settings” this will reveal the interface for setting the proxies.
You should use the proxy credentials you got from your provider. Currently, Octoparse only supports HTTP proxies and as such, that is the proxies you should use.
4. Click OK and then Save and your proxies would be saved.
What is Octoparse?
Octoparse is a visual web scraper developed to be used by those without coding skills. This web scraper is built for the modern web and you can use it to extract data from any website of your choice provided the data you want to scrape is publicly available online.
Octoparse has got support for Javascript rendering. With this web scraper, you can turn data points on hundreds of thousands of pages on a website into a structured spreadsheet with just a few clicks. Some of the other features of this tool have included support for scheduled scraping and even cloud scraping, among others.
Why Use Proxies for Octoparse?
Octoparse does not force you to make use of proxies and even without proxies, you can use still make use of it. However, you will get blocked after a few attempts as the requests it would send will all have the same IP footprint which would reveal to your target sites that you are using a bot as the number of requests it sends is unnatural. With proxies, you get many IP footprints, thereby making it difficult to detect you are using a bot.
- Get best proxies for Phantombuster
Rotating Vs. Sticky Proxies - Which are the Best for Octoparse?
Octoparse is a web scraping tool and as such, for most cases, the best proxies to use are rotating proxies. You can go ahead and use a bunch of private proxies and the tool will rotate them for you.
However, using rotating residential proxies are the best since they are cheaper when you need thousands of proxies. However, if the tasks you need to carry out require you to scrape just a few pages, then using private proxies would be the cheaper option for you.