Recently I upgraded my firewall and decided to go the custom route. My flavor of choice was PFSense, as it has robust features. After months of using it, I couldn’t be happier, but there was one issue I didn’t like, I couldn’t configure the firewall to block bot/port scans. A few google searches produced nothing. People mentioned using Snort, but as I started playing with those tools, I thought there must be a simpler way. To give you some background on my motivations and why I do this: when I first started building cloud servers, I was shocked by the amount of traffic that is constantly bombarding your firewall. I saw quite a bit of repeat offenders. So on my cloud instances, I used UFW + fail2ban to block anything that touched ports outside my “allowed ports”.
FYI - this can be super frustrating when you “forget” that you have this set and start playing with something, like a web server on a different port. Depending on your cloud provider, you could be screwed (I somehow blocked everything including localhost once and had no console, YIKES!). You may need to connect via VPN in case of emergency. I also recommend taking an audit for several days just to make sure that legit services are not reconfigured, as you do not want to block them.
My home network has nowhere near as many public facing services as my cloud servers, but it does have some and that makes me nervous because it is where literally EVERYTHING resides. Most of the traffic you see is bot scans and usually only a few ports. From time to time, you'll get someone with some talent who is curious. Its that person I'm concerned about, because when they find your only open port[s] and begins to test, all they have to do is wait for you to forget to patch something and then they’re able to use a near-day to get a foothold. The thought of this makes me wake up in cold sweats! BTW - explaining this to people outside the community makes feel as if I should be wearing aluminum foil. Like with my cloud servers, blocking an IP from touching any port outside your allowed range makes it SIGNIFICANTLY harder to scan and see whats open. For the record, it can still be done. You could use something like TOR to funnel the port scan through, then use the TOR management port to change IP’s for each port. However that would only give you around 7k ports as there are only about that many TOR endpoints. So you'd eventually get an nmap top 1k but not a full port scan, as there are not 65535 exit nodes. I did read a writeup way back that said sneaker proxies (yes, as in "shoe" proxies) have access to large ranges of unique RFC 1918 IP address if you REALLY were diligent on getting a full scan (*cough* compromised machines with RAT's installed to proxy connections though *cough*).
Anyhow, I didn’t want to go with my cloud solution and “corrupt” PFSense with something like fail2ban, as PFSense is a premium firewall and toolset. After doing some digging into how PFSense firewall does it's logging, I realized that my “catch all REJECT” rule located at the bottom of my WAN interface ruleset had the option for logging. I turned that on and located the log location on the filesystem. PFSense logs in CSV, which makes it super easy to pull and filter logs. I dumped my “/var/log/filter.log” and a novel worth of information dropped out:
Thinking on my feet, I logged onto one of my cloud servers, installed netcat and attempted to connect to a port that was not on my “approved” list, then grepped for the cloud server IP address in my PFSense logs:
grep 'IP.ADDRESS' /var/log/filter.log
Wow, that was pretty simple, I had found it in the first go, this was going to be a good morning. It produced a wealth of information, as the IP address was associated with other rules I had created. So I went back into the PFSense firewall config for that specific rule to see if there was some kind of label for it, and conveniently enough, there was! This picture depicts where to go to find the rule number:
A note: You’ll notice that I do not have my rule set to “drop”. I recently move to the school of thought that it is better to let the person or bot know right away that the port is blocked instead of forcing them to wait many more milliseconds to time out. Reject is nearly immediate where as they'll continue to wait for a packet to come back, so it creates far fewer states on your router. There is a really good writeup on what I am talking about here here (http://www.chiark.greenend.org.uk/~peterb/network/drop-vs-reject). It will save you some resources!
ANOTHER note on this: I still use “drop” on certain cloud servers, especially during red team engagements I do this because I want full anonymity. Sometimes security minded people, or people like me with an aluminum foil hat on staring at their logs all day, will sometimes counter scan you. With blocking, if the said person goes to do a counter scan and nothing seems to be there, they may assume "Hmmm, maybe it was a docker image that spun up for a scan or maybe they are behind a firewall that is doing NAT, and they may just forget about you. If you want want that level of anonymity, I totally understand and uses that level sometimes. For me and my home network, resources are king. I cant stop building things and because of that am now would be in the run for a home-enterprise sized network of the year. So I need to save resources where I can.
Depicted above, “Tracking ID” showed up in the logs when I grepped for the number associated with it. Now I had to find a way to pull the IP information, get them into some kind of crude file or database and make sure that there were no duplicates (I later realized that PFSense would handle duplicates for me, but my script does it also). I initially created a bash script that pulled the information. It was stupid simple and took me maybe 10 mins to write. However, I felt it was very “flimsy” and I was not at all confident with letting a flimsy, half-assed shell script handle my banned IP’s list. For the record, there are more "flimsy" bash scripts running than I'd like keeping me afloat. But this is one that could lock me out completely. What do I mean by "flimsy? I guess I'll share:
#!/usr/local/bin/bash
grep 'RULEID' /var/log/filter.log |cut -d ',' -f 19|sort|uniq >> /root/Initial.txt
cat /root/Initial.txt|sort|uniq > /root/IPBlacklist.txt
cp /root/IPBlacklist /root/Initial.txtYeah, I didn’t like it either. So I warmed up my python skills and created something that made me feel warm and fuzzy inside:
#!/usr/local/bin/python2.7
import os
import time
DBsave = "/root/IPBlacklist.txt" #Change if you don’t like the location
def Filter(data):
count = 0
path = "/var/log/filter.log"
f = open(path,'r')
olddata = []
unsorted = []
with open(data) as old:
for odata in old:
if len(odata.strip()) != 0:
olddata.append(odata.rstrip("\n"))
sorted = []
with open(path) as text:
for line in text:
if 'RULEID' in line: #change this line based on your rule number
unsorted.append(line.split(',')[18])
nextdata = list(set(unsorted))
nextolddata = list(set(olddata))
for nextline in nextdata:
if nextline not in nextolddata:
count += 1
sorted.append(nextline)
if count == 0:
exit()
else:
for fileline in sorted:
file = open(data, 'a+')
file.write(fileline + "\n")
file.close()
if not os.path.exists(DBsave):
with open(DBsave, 'w'): pass
Filter(DBsave)Here's a direct link to my github in case the blog software gets the spacing wrong: https://github.com/s0meguy1/PFBlock/blob/master/blacklist.py By no means would I call this a world-renowned python script, but it made me feel much better, and I could later do things like add timestamps, error handling (which I did none of here) or whatever came up as things started breaking. The sky’s the limit with Python.
This was great and all, but I realized I had no way to get this newly created list into PFSense. Luckily, I had installed PFBlockerNG, which I HIGHLY recommend! I also use it as a quasi-pihole, and block entire countries or regions from accessing my home network (don't worry, google hosts this blog. I wanted a larger audience than my local city). PFBlocker has the option for custom FQDN whitelists, but also IPV4 (and IPV6) whitelists. This feature is usually used to pull from URL’s which contain large lists of IP addresses that people are constantly updating, which I also do, But I decided to see if it would read from the filesystem, which it did.
To do this, within the PFSense web app (with PFBlocker installed), head to:
Firewall -> PFBlocker -> IPv4
From there, you add a new list, give it an alias (alias might be optional? I can’t remember), then under “IPv4 Lists” you can add the blacklist from the absolute path of the txt file you created in the python script (Ea /location/of/your/sript.txt – the DBsave variable). I set my list to update every hour:
Fantastic! But one more thing, you must add the python script to cron so that it can be set to run and update. As I said before, this could be done within the python script, however I was too lazy to start down that path when I could just have cron do it for me. Also it probably would eat more resources to do that. I decided since the PFSense list was updating every hour, that I would have cron do it every 30 minutes just to be safe, so I ran “crontab -e” and added:
*/30 * * * * /usr/local/bin/python2.7 /root/blacklist.py*** you may want to verify your version of python is in /usr/local/bin/
And that’s it! You are now preventing unwanted harassment from returning to your network! If you're interested in what is next, come back and checkout my blog! or follow me on twitter: @s0meguy1 I'll be posting this there as well. Good luck!