I.P Blocklists?
- Thread starter Cyber-Mav
- Start date
The feeds are presumably a list of known botnet c&c servers, so preventing your clients reaching out to them can be helpful. Same with geo restrictions - if you never think you're going to need to connect to a Russian website then dropping outbound connections is a cheap way to add a very incremental level of security.
Last edited:
You can also use lists to block ad and tracker servers, using Unbound (built into *sense) or via plugins like AdGuard Home. You didn't specify what the lists were blocking, so I thought I'd add this in. Blocking almost all ads and trackers at the DNS level is not just useful, it's a must imo.
Soldato
Depending on which sense you are using. There is a great YouTUbe channel by Tom fro mLawrence Systems and he does a lot of pfSense walk throughs. His pgBlockerNG one is really good, shows you how to set it up and hints at some of the normal blocklists built in you want to utilise.
I have a few DNS block lists in the DNS blocker of pfblockerng but turn DNS blocking off as it causes mayhem with my Mrs as she googles something then clicks the sponsored link EVERY TIME and it generally blocks those.
HaGeZi's lists block everything you'd expect, but Gerd (the maintainer) has them very well curated. Click-through shopping links and affiliates etc all work fine, but they block a *lot* of crud. The lists are intended to work alone (you don't need this list and that list), and I'd recommend you try Pro. You can add TIF (Threat Intelligence Feed) for more complete coverage against malware, botnets and C&C servers etc, but it isn't essential.
I've run his lists for a long time (well over a year, maybe two) on my family network - including DoH profile for offsite usage on phones etc - and haven't had a single complaint from the wife. There's a comparison to OISD at the bottom of the page, which shows even his Light list blocks a lot more ads/trackers but the FP rate is basically zero.
Associate
It's a shame gerd's lists arn't replicated for hosts usage!
Last edited:
Sorry, I'm not sure what you mean?
Associate
Last time I wanted to use some of the lists, IE what Feek posted are/was not available in HOST format.
Edit: this https://forums.overclockers.co.uk/threads/setting-up-pi-hole.18756115/page-52#post-37310718
Edit: this https://forums.overclockers.co.uk/threads/setting-up-pi-hole.18756115/page-52#post-37310718
Last edited:
Associate
Yeah no change, so if you check the adblock folder vs the hosts folder options are meagre in comparrison. unless of course I have misunderstood and the lists are all compiled together in host format,
Last edited:
The Adblock lists are more granular, if that's what you mean. That's as much a factor of their flexibility as anything, because you can do more with them. Stopping this from turning into an XY problem, what are you actually missing in hosts format? All the main block lists are available, as well as native tracking extensions. All you need is Pro (or whatever level) and you're away? Or are you needing the anti-piracy and similar lists? What blocker are you actually using? Swap to one that can accept Adblock format or RPZ if you need more flexibility.
Edit: If there's a list you really want in hosts format, just open a ticket. I'm sure Gerd will help you out.
Edit: If there's a list you really want in hosts format, just open a ticket. I'm sure Gerd will help you out.
Last edited:
Associate
I wanted to test like for like adgaurd lists vs hosts format but it seems I can't, and also keep my lists smaller, problem being is I only have 500mb to play with for my lists.
Mikrotik load lists directly into memory. As you can see 2 lists sucks up quite a bit of memory.
So the cost for my router is around 200MB seeing as the pro list only contains part of the tif list for example. The router only does hosts files at this point.
Anyway, thanks.
Mikrotik load lists directly into memory. As you can see 2 lists sucks up quite a bit of memory.
So the cost for my router is around 200MB seeing as the pro list only contains part of the tif list for example. The router only does hosts files at this point.
Code:
0 url="https://raw.githubusercontent.com/hagezi/dns-blocklists/main/hosts/pro.txt" match-count=15847 name-count=508159
1 url="https://raw.githubusercontent.com/hagezi/dns-blocklists/main/hosts/tif.txt" match-count=2 name-count=1241261
/ip/dns/print
servers:
dynamic-servers:
use-doh-server: https://cloudflare-dns.com/dns-query
verify-doh-cert: yes
doh-max-server-connections: 2
doh-max-concurrent-queries: 100
doh-timeout: 6s
allow-remote-requests: yes
max-udp-packet-size: 4096
query-server-timeout: 2s
query-total-timeout: 10s
max-concurrent-queries: 100
max-concurrent-tcp-sessions: 20
cache-size: 250000KiB
cache-max-ttl: 1w
address-list-extra-time: 0s
vrf: main
mdns-repeat-ifaces:
cache-used: 192160KiBAnyway, thanks.
I get you. In your position I'd run DNS on its own device, like a Pi or local server. Squeezing a lot of blocklists onto an embedded router is a recipe for frustration and slowdowns. I'm old school, but I think those types of services should be independent anyway rather than all-in-one. You could always ask Gerd to make you a TIF mini in hosts format, if you really want to do a comparison. TLDR though, Adblock format lists will block everything the hosts does (and more) despite being a smaller list. It's simply a far superior format.
Soldato
I'll give it a blast, I'll silently slip it on whilst the family are in bed and if anyone moans I'll plead ignorance.
Thank you
@Steveocee Do it, you'll be fine. Worst comes to worst you can quickly whitelist the offending domain and plead ignorance lol The Pro is a very nicely balanced list; not as aggressive as Pro++ (which itself is fine tbh) but does catch everything you'd expect. Even the Light does a bang up job, and would be sufficient - it catches a lot more than OISD for example.
@Ad_Augendae You said you 'only' have 500MB to play with for lists. I've just logged into our two DNS servers and they're using 204MB and 210MB of RAM for AGH (under 250MB all in for the whole system). That's basically 200MB for the AGH binary itself, logging, Hagezi Pro, TIF Full, TIF IPv4 list, Dandelion Sprout's Anti-Malware List, optimistic caching enabled, and 10MB of DNS cache - all in memory. You should be able to manage fine, provided your blocking software isn't a bloated mess.
@Ad_Augendae You said you 'only' have 500MB to play with for lists. I've just logged into our two DNS servers and they're using 204MB and 210MB of RAM for AGH (under 250MB all in for the whole system). That's basically 200MB for the AGH binary itself, logging, Hagezi Pro, TIF Full, TIF IPv4 list, Dandelion Sprout's Anti-Malware List, optimistic caching enabled, and 10MB of DNS cache - all in memory. You should be able to manage fine, provided your blocking software isn't a bloated mess.
Last edited:
Associate
You should know I like to test stuff by now. So I've been using HaGeZi's lists for about 2 years but mostly on Rpi4 8GB DoH enabled via Cloudflared Proxy et al and never needed to worry. Testing with a list about 1.7 Million causes a slight bog-down, not terrible but none the less slower. I believe in your words Adguard lists are more efficient/smaller compressed ? My normal list 'pro' uses about 60/70MB (the whole list gets dumped into memory for speed I guess!) but there is no simple whitelist function on the Mikrotik platform I am using atm which gives me DoH with adblocking all in one place with less hardware to worry about with the pi being a backup dns as well. long term plan is to get an RB5009 or later edition in about 12 months as the Mikrotik WiFi has matured and working as expected for me at least. You know I've tried all the adblockers at some point or other. The TIF list was a curiosity to see if I could get away with it, but for now the pro list works very well.