I have nothing against proxies. They have many legitimate uses. I wrote a script which scrapes xroxy.com and provides a plain text proxy list for the Firefox SwitchProxy extension. But I run a website too, and part of managing the community means dealing with members who evade bans or obscure their identity through the malicious use of proxies. I was frustrated that there is no reliable way to detect if someone is using a proxy based on their IP, so I came up with my own solution, the Black Box Proxy Block.
The goal is to detect all open HTTP, SOCKS, and web-based proxies, TOR exit nodes, dedicated servers, and public VPNs. Of course that's not really obtainable, but I'm getting closer all the time. When I discover a proxy that was missed I try to figure out how to automate its detection in the future, and some of our members seem very determined to help with the process. :) After nearly a decade in service on a site with 1 million+ unique visitors a month, it has become fairly reliable.
To test the Black Box Proxy Block, just enter an IP address here:
Integrating the service into your own site is easy. When you want to check if someone is using a proxy, simply request:http://www.shroomery.org/ythan/
The response is a single character and will contain one of three values: Y if it's a proxy, N if it isn't, or X if there's an error.
This service only detects open proxies and dedicated servers. You should also consider using Stop Forum Spam and BotScout to check for known bots and spammers.
If you have any feedback, questions, suggestions, or want to report false positives or false negatives, please contact me.
Frequently Asked Questions:
Q: Can I have a copy of your database, or run the script locally?
A: I'm very sorry, but no. That's where the "black box" part of the name comes from. ;) This is first and foremost a service for my own use. The public interface was just an afterthought. If you can benefit from it too, awesome! I plan to keep it online and freely available for the foreseeable future. But I don't want to compromise its effectiveness by giving away details about how it works. I believe security through obscurity has its place, and I'd rather not offer proxy admins any advantages when it comes to preventing detection.
Q: Can you help me integrate proxy blocking into my site?
A: If you want to hire me as an independent contractor, sure! Drop me an e-mail and we can discuss specifics. Otherwise, I'd recommend posting on the official message board for the software you're running, and see if someone with experience can help you. Most complex scripts already have methods in place for retrieving remote files and caching results, so it shouldn't be too difficult for someone who's familiar with the code.
Q: Do you have any plans to offer this service in the form of a DNSBL?
A: I'd love to. I understand it would be more convenient in a lot of situations. I tried to set it up once, but ended up breaking BIND. Unfortunately I don't have lots of free time to spend on this project, but if anyone has experience running an RBL and wants to help me out, please get in touch.
Q: Are there any other similar services out there?
A: Yes, although I've found them to be deficient in one way or another for my own personal needs. But you may wish to check out BlockScript, BlockThatProxy, MaxMind, ThreatMetrix, Proxy IP Checker, or Nasty Hosts. There are also some relevant DNSBLs like tor.dnsbl.sectoor.de, proxies.dnsbl.sorbs.net, and bl.blocklist.de.
Q: Is there a limit to how much I can use this service?
A: No, there's no limit, although when you check an IP I'd appreciate it if you cache the result to eliminate redundant requests. However, please make sure your own code fails gracefully if it receives unexpected input, or if the proxy check times out. This site uses CloudFlare, so if you somehow trigger their DoS protection, you'll be presented with a CAPTCHA instead of the results you were expecting. Also, sometimes we just experience good old-fashioned downtime. Make sure you plan accordingly, so if I screw up, it doesn't break your site. ;)
Q: Is there a business-class version of this service?
A: Not at the moment. I'm considering offering a premium version, but I'm not sure if anybody would be interested. Obviously, the free version will always remain the same and will continue to be updated. However, for a modest fee, we could provide real-time checks using a variety of IP reputation sites, and perform additional screening by attempting to connect to the IP to see if it's an open proxy. We could offer this service in the form of a DNSBL, or over HTTPS for subscribers. We could also provide a dump of our current IP blacklist for subscribers who wish to perform screening offline. Of course, we still couldn't guarantee 100% successful detection with no false positives, but it would increase the effectiveness of the service to the same level that we use internally on our own sites. Is this a service you might be willing to pay for? What do you think would be a fair price? Please let me know if it's worth the effort!