The Top 7 Free Proxy Lists for Web Scraping
Robert Munceanu on Aug 16 2021
In web scraping, you’re either using proxies or doing a poor job extracting data. Sure, you may not need a backconnect proxy with thousands of datacenter and residential IPs, but you’ll at least want some basic proxies.
The big problem for hobby or beginner scrapers is that IPs usually cost money. Luckily, there is such a thing as free proxies. In fact, there are quite a few options to choose from. We’ve explored some of them and chose the best 7 so that you only get the quality, at least as far as free proxies go.
Actually, before we get into the list, we should go over some details. Free proxies are great because you don’t have to invest any money, but they aren’t perfect.
The pros and cons of free proxies
First and foremost, they’re free. That’s a bit obvious, but it’s a significant advantage. Just think, you can take any one of the guides on our blog, code your scraper, and then use it together with the proxies.
There is much to say about the benefits of proxies in web scraping, and we won’t go over them in this article. But, if you want to know more, check out this article on rotating proxies, in which we present just how big of a difference an IP pool can make.
Sadly, that’s pretty much the only advantage that free proxies have over others. Now, onto the cons.
Since these IPs are freely available, everyone knows about them. So, websites likely already know the addresses and have already blocked them. Either someone already tried to access the website you’re interested in and got the IP banned, or the webmaster found the free proxy provider, copied all the IPs, and added them to a blacklist. This is not always the case, but it happens often.
Free IPs are often slow and lack anonymity. Maintaining proxies costs money, so you can’t seriously expect to get top-quality IPs at no cost. As the saying goes, you get what you pay for.
Free proxies may expose you to hackers. As I just said, it’s a bit unusual for someone to offer free addresses for nothing. One nefarious reason someone may do this is to monitor and copy the information that passes through the proxy. After that, they may use that information for malign purposes. Again, this isn’t a guarantee, but you should stay on guard.
The one big factor that sets WebScrapingAPI apart from other providers on this list is that the API only uses premium proxies. That includes both residential and datacenter IPs, with plenty of geolocation options (7 for datacenter, 40 for residential, and over 200 for custom plans). While there are paid plans, ranging from $20 to $200, any new user receives a free trial period of two weeks, time in which they can use any of the API’s features.
Proxy rotation is handled automatically by the API so, unless you specifically ask otherwise, all requests will go through different IPs. Moreover, you can send up to 10 concurrent requests, meaning that you can use the proxy pool to its full extent, extracting data at breakneck speeds.
Naturally, WebScrapingAPI doesn’t offer only proxies but also access to the scraper. So, you get all the features of a premium data extraction tool without writing a line of code.
Once the free trial is over, you still don’t have to spend a dollar unless you want to because you’ll be downgraded to the free package, with 1000 API calls per month for as long as you want them.
ProxyScan is a collection of useful tools rolled into a single website. First and foremost is the proxy list. They claim to have over 12.000 proxies in total. While it’s an impressive number, from what I could glean, the number might be a bit lower, possibly because free proxies are constantly appearing and disappearing, so the number may vary.
Of those proxies, almost all are SOCKS 4 or 5, with just over 100 being HTTP/S. Also, nearly all are anonymous or elite, which are the only ones that really matter in web scraping.
On the subject of geolocation, ProxyScan offers IPs from over 100 countries. While not all of these will have a large number of proxies, it’s still good coverage. You can also check every IP’s uptime and ping, making it easy to pick the best IPs.
The general idea behind Proxy-list seems to be quantity over quality. In total, they have over 17.000 proxies, most of which are SOCKS 4. I say that there’s less focus on quality because you don’t get any info on the IP’s uptime or ping. The website updates its lists every two hours and eliminates unresponsive IPs.
When looking at their lists, you get the IP, the port, the anonymity level, and the country. Compared to other providers on this list, that is somewhat lacking.
You can filter results by anonymity (transparent, anonymous, or elite) and by country. I’ve noticed that some of the countries you can select in the filter don’t actually have any proxies available on the website. So, don’t necessarily expect to find IPs from any region. That being said, there are still plenty of options.
Besides browsing through their many pages, you can also download lists as plain text or straight to the clipboard. While the process is a lot faster like this, the txt file only has the IPs, so if you want to also record the ports or countries of origin, you’ll have to get them straight from the site. As you’d expect, I suggest you scrape the pages instead of getting the data manually.
4. Free Proxy
While Free Proxy may not be the best-looking website, it has a vast number of IPs, their details, and all the filtering functionalities you’ll need.
The total number of IPs exceeds 23.000, most of which are SOCKS4/5. These IPs are gathered from a total of 160 counties. Most IPs are from Asia, but you’ll have little problem finding proxies from any corner of the world.
If you look at their lists, you’ll see a wealth of information, including speed, uptime, response, and how much time has passed from the last check. That last detail is actually one of the weak points for this provider - many IPs are checked quite rarely, and there’s a good chance to run into many dead IPs, especially if it’s been days since their last ping.
To navigate, you can filter by country, anonymity, and protocol. Then, you can also order the resulting list by speed, uptime, response, and “last checked”. So it shouldn’t take long to find what you’re looking for. Also, there’s a button named “Export IP:Port” that lets you quickly copy the whole address so you can add it to your scraper.
Like the previous provider, Spys.one offers a wide range of IPs from all over the globe. In total, they seem to have more than 26.000 proxies, so it surpasses Free Proxy. But many IPs haven’t been checked in weeks or even months, so they may not work anymore.
Navigation-wise, you can choose just about any criterion to filter the list. You can even select by city, even though plenty of proxies don’t actually have a specified city. So, use that option only if it’s crucial for your use case. Otherwise, you might gloss over valuable possibilities.
While the provider may seem to have a large number of IPs, I can’t say I’m too fond of the UI, so getting all those IPs might be a bigger hassle than you’d expect.
Unlike most other websites on this list, Genode doesn’t make money from ad revenue but by renting premium residential proxies. But besides those, they also maintain a free proxy list that any visitor can use.
Their list contains almost 5.000 proxies spread across an impressive number of countries. To navigate this list, you can choose the country, level of anonymity, proxy protocol, organization, speed, uptime, and last check date. So essentially, you filter by all the differentiators they present.
When checking speed, you get an actual time frame (in milliseconds) and a bar that compares the speed of that particular IP to the others in the list. Also, if the response time is abnormally large, the number will appear in red, signifying that it’s probably too slow for proper use.
These kinds of small details can have an impact on your overall results, so Genode deserves a spot on the list even if the proxy pool itself is a tad small compared to the other website I presented.
7. Free Proxy Lists
While I can’t call Free Proxy Lists the most beautiful website, it definitely cuts to the chase, presenting you with an extensive list of IPs.
It’s commendable that the platform has proxies from 78 different countries, a number that you might not get from paid service providers. Unfortunately, many of these countries have less than 10 IPs each. In many cases, there’s a single proxy from the region.
In total, they have over 600 IPs, all of which use the HTTP or HTTPS protocol, so you’ll have no luck if you need SOCKS.
Simply passing from page to page looking for the right proxy can get boring quickly, but the website lets you filter results by country, port, protocol, anonymity, and uptime. You can also see their response and transfer speeds represented by colored loading bars. While not very explicative, they can help choose the faster IPs.
Choose your proxies wisely
While I understand the appeal of free proxies and recommend you at least try them, remember that your time is also a resource. In fact, it may be more valuable than money.
Let me elaborate: cycling through thousands of free proxies may cut costs since you’re not playing for premium proxies, but it will require constant attention from you. The IP will often stop working or get blocked, and you’ll then have to find a new list and add it to your script. As new free IPs appear, you’ll need to add them since the old ones will sooner or later go down.
In essence, free proxies save money but require some repetitive work. I’d recommend any scraping enthusiast builds at least one scraper and uses it with free proxies because it’s an excellent learning experience. After that, though, you may just want accurate data delivered on time, with little to no human intervention.
That’s the thought process behind WebScrapingAPI. We’ve built a REST API to do most of your work for you so that you can concentrate on actually using data to its full extent. If you haven’t already, start your free WebScrapingAPI trial to get your hand on a top-notch scraper with airtight proxies!