DNS blackhole using FreeBSD

# pkg search bind9
bind914-9.14.9                 BIND DNS suite with updated DNSSEC and DNS64
# pkg install bind914-9.14.9
# rndc-confgen -a
# sysrc named_enable=YES

The majority of blocklist are in ‘hosts file’ format, ie they specify an IP address (typically 127.0.0.1) and then FQDN. We need to modify these files into bind format so that we can use them as zone files and return the relevant address information.

We will use the hosts files collected by firebog.net which themselves come from various sources:

# https://v.firebog.net/hosts/lists.php?type=tick
https://hosts-file.net/grm.txt
https://raw.githubusercontent.com/PolishFiltersTeam/KADhosts/master/KADhosts_without_controversies.txt
https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Spam/hosts
....

I had originally wanted to write a bash script to download the host blacklist and munge them into bind zone files in an effort to present a solution with very few dependencies. However the host files came in a variety of formats requiring some pretty basic regex which were not possible with just BRE and ERE regex support in the bash shell. So I compromised and wrote some of the code in Python but with the imposed mandate of using any libraries not in the base distribution.

First off, install git so you can clone my repo:

# pkg search git
# git-2.24.1                     Distributed source code management tool
# pkg install git

Now clone the repo and take a look at the code:

# cd ~
# mkdir dnsbl-bind
# git clone https://github.com/sebrupik/dnsbl-bind.git

Python3.7 is part of the FreeBSD base install so the only additional package we need is wget, then the script is good to run:

# pkg install wget
# cd dnsbl-bind
# chmod +x dnsbl-bind.sh

Now before you run the script the named.conf file needs to be edited. We need to add some comments to the file which the script will use are markers to insert configuration items.

# /usr/local/etc/namedb/named.conf
...
options {
...
//options++//
//options--//
};
...
//zones++//
//zones--//
...

The shell script needs to be run as a user who can write to the /usr/local/etc/namedb/ folder. The script initially builds the required folder structure and then retrieves the various hosts files from v.firebog.net . The host files have several different formats and comment styles so a python script is run to extract and unify the data before outputting into a format (RPZ) which bind can understand. named.conf is then edited to reference these RPZ files.

Now run the script:

# ./dnsbl-bind.sh
...
--2020-05-13 23:11:14--  https://zerodot1.gitlab.io/CoinBlockerLists/hosts_browser
Resolving zerodot1.gitlab.io (zerodot1.gitlab.io)... 35.185.44.232
Connecting to zerodot1.gitlab.io (zerodot1.gitlab.io)|35.185.44.232|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 90237 (88K) [text/plain]
Saving to: 'hosts_browser'

hosts_browser       100%[===================>]  88.12K   416KB/s    in 0.2s    

2020-05-13 23:11:15 (416 KB/s) - 'hosts_browser' saved [90237/90237]

FINISHED --2020-05-13 23:11:15--
Total wall clock time: 1m 42s
Downloaded: 32 files, 11M in 6.4s (1.77 MB/s)
5
got some arguments!
no regex match for: Malvertising list by Disconnect
 in simple_malvertising.txt
no regex match for: mosumumopo
 in Shalla-mal.txt
We parsed 32 input files
We parsed unique 381616 domains
Writing FI8LUZ, source KADhosts_without_controversies.txt with 10814 items
Writing PERY3Q, source hosts with 73 items
Writing QDM1AD, source w3kbl.txt with 773 items
Writing CNZFN1, source hosts.txt with 12129 items
Writing INQF6U, source AdguardDNS.txt with 34690 items
Writing OOXA0K, source adservers.txt with 38284 items
Writing 2AGFGF, source simple_ad.txt with 1675 items
Writing D3M1KC, source Easylist.txt with 745 items
Writing N68PHF, source serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext with 555 items
Writing CTJXP1, source hosts.1 with 9 items
Writing ORIN8D, source hosts.2 with 8092 items
Writing XUELF7, source Easyprivacy.txt with 839 items
Writing 0KOT4U, source Prigent-Ads.txt with 744 items
Writing OE2XZH, source notrack-blocklist.txt with 12006 items
Writing 6I5X4Z, source hosts.3 with 1214 items
Writing 16CBSC, source spy.txt with 286 items
Writing EHIKTD, source ads-and-tracking-extended.txt with 103450 items
Writing JALC4F, source AntiMalwareHosts.txt with 122 items
Writing AQNTMV, source firstparty-trackers-hosts.txt with 36029 items
Writing SSKK7J, source latestdomains.txt with 597 items
Writing 7KSLT6, source simple_malvertising.txt with 3 items
Writing 6L5EY9, source justdomains with 26818 items
Writing H8ZWFK, source Prigent-Malware.txt with 42328 items
Writing HZGWZF, source immortal_domains.txt with 2628 items
Writing NKMELK, source hosts.txt.1 with 1007 items
Writing NCUCHP, source phishing_army_blocklist_extended.txt with 13030 items
Writing 0W06Q6, source notrack-malware.txt with 270 items
Writing WJVE0R, source Shalla-mal.txt with 19123 items
Writing II4IR8, source main-blacklist.txt with 8066 items
Writing U2E46V, source hosts.4 with 2021 items
Writing D1BR4V, source index.html with 321 items
Writing TC3FQJ, source hosts_browser with 2875 items
#

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at WordPress.com.

Up ↑

%d bloggers like this: