Setting up Pi-hole

Ad_Augendae · 10 Sep 2021 at 14:28

Not sure if this is the best place to post but I keep getting timeout errors from cloudflared DOH Proxy using pi-hole+cloudflared
Used these settings https://docs.pi-hole.net/guides/dns/cloudflared/
I don't seem to have any heavy requests at the times it fails, couple of times a day mainly. Just curious if anybody else get these or not.
RPI 4/64 8GB
cloudflared -v
cloudflared version 2021.8.7 (built 2021-08-28-1752 UTC)
Proxy
pihole -v
Pi-hole version is v5.3.1 (Latest: v5.3.1)
AdminLTE version is v5.5.1 (Latest: v5.5.1)
FTL version is v5.8.1 (Latest: v5.8.1)

Errors:

Code:

Sep 10 10:03:26 raspberrypi cloudflared[712]: 2021-09-10T09:03:26Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="failed to perform an HTTPS request: Post \"https://1.1.1.1/dns-query\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 10 10:48:10 raspberrypi cloudflared[712]: 2021-09-10T09:48:10Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="failed to perform an HTTPS request: Post \"https://1.1.1.1/dns-query\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 10 10:48:11 raspberrypi cloudflared[712]: 2021-09-10T09:48:11Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="failed to perform an HTTPS request: Post \"https://1.1.1.1/dns-query\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 10 10:48:14 raspberrypi cloudflared[712]: 2021-09-10T09:48:14Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="failed to perform an HTTPS request: Post \"https://1.1.1.1/dns-query\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)

This seems to be an ongoing thing https://github.com/cloudflare/cloudflared/issues/91
time dig +short google.com @127.0.0.1 -p 5053
142.250.13.139
142.250.13.101
142.250.13.113
142.250.13.102
142.250.13.100
142.250.13.138

real 0m0.060s
user 0m0.040s
sys 0m0.020s

Ad_Augendae · 10 Sep 2021 at 16:41

Umm anybody tried this --max-upstream-conns flag, seems to be an arm issue I think. If so what settings for rpi4 20 ish! it was added for a reason I guess.
https://github.com/cloudflare/cloud...19e6bf856/cmd/cloudflared/proxydns/cmd.go#L56
More dross here https://github.com/visibilityspots/dockerfile-cloudflared/pull/35

Iv'e adjusted mine to 10 and will report back, this is for RPI 4/64 8GB

Code:

sudo nano /etc/default/cloudflared
CLOUDFLARED_OPTS=--port 5053 --upstream https://1.1.1.1/dns-query --upstream https://1.0.0.1/dns-query --max-upstream-conns 10
sudo systemctl restart cloudflared

Rainmaker · 10 Sep 2021 at 18:12

@Ad_Augendae I can't speak for Cloudflare, and as you say it seems to be a connection limit you're hitting. Try with Quad9 instead? https://dns.quad9.net/dns-query for standard encrypted DNS (with DNSSEC), plus malware/ransomware/c&c server filtering.

Ad_Augendae · 10 Sep 2021 at 20:43

Thanks I'll have a look at that if this starts to fail again.
I have add blocking on the router (Steven Blacks List) but I've had to fine tune some sites and I can't white list on the router, pi-hole is great as well but I just need too iron out a few little quirks. I'm still testing stuff and as yet not 100% settled on what combination to use security wise. I did read a few negative issues with https not being all it's cracked up to be, but is anything these days!

Ad_Augendae · 10 Sep 2021 at 21:05

Hahaha would you believe, just checked again and it's failed a few mins after posting.

Code:

Sep 10 17:41:56 raspberrypi cloudflared[16159]: 2021-09-10T16:41:56Z INF Adding DNS upstream url=https://1.1.1.1/dns-query
Sep 10 17:41:56 raspberrypi cloudflared[16159]: 2021-09-10T16:41:56Z INF Adding DNS upstream url=https://1.0.0.1/dns-query
Sep 10 17:41:56 raspberrypi cloudflared[16159]: 2021-09-10T16:41:56Z INF Starting metrics server on 127.0.0.1:37159/metrics
Sep 10 17:41:56 raspberrypi cloudflared[16159]: 2021-09-10T16:41:56Z INF Starting DNS over HTTPS proxy server address=dns://localhost:5053
Sep 10 20:49:04 raspberrypi cloudflared[16159]: 2021-09-10T19:49:04Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="failed to perform an HTTPS
Sep 10 20:49:04 raspberrypi cloudflared[16159]: 2021-09-10T19:49:04Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="failed to perform an HTTPS
Sep 10 20:49:05 raspberrypi cloudflared[16159]: 2021-09-10T19:49:05Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="failed to perform an HTTPS
Sep 10 20:49:05 raspberrypi cloudflared[16159]: 2021-09-10T19:49:05Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="failed to perform an HTTPS
Sep 10 20:49:07 raspberrypi cloudflared[16159]: 2021-09-10T19:49:07Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="failed to perform an HTTPS
Sep 10 20:49:07 raspberrypi cloudflared[16159]: 2021-09-10T19:49:07Z ERR failed to connect to an HTTPS backend "https://1.1.1.1/dns-query" error="failed to perform an HTTPS

Lets have a look tomorrow.

Code:

sudo systemctl status cloudflared
● cloudflared.service - cloudflared DNS over HTTPS proxy
   Loaded: loaded (/etc/systemd/system/cloudflared.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2021-09-10 21:12:43 BST; 6s ago
 Main PID: 17964 (cloudflared)
    Tasks: 8 (limit: 4915)
   CGroup: /system.slice/cloudflared.service
           └─17964 /usr/local/bin/cloudflared proxy-dns --port 5053 --upstream https://dns.quad9.net/dns-query

Sep 10 21:12:43 raspberrypi systemd[1]: Started cloudflared DNS over HTTPS proxy.
Sep 10 21:12:43 raspberrypi cloudflared[17964]: 2021-09-10T20:12:43Z INF Adding DNS upstream url=https://dns.quad9.net/dns-query
Sep 10 21:12:43 raspberrypi cloudflared[17964]: 2021-09-10T20:12:43Z INF Starting metrics server on 127.0.0.1:35933/metrics
Sep 10 21:12:43 raspberrypi cloudflared[17964]: 2021-09-10T20:12:43Z INF Starting DNS over HTTPS proxy server address=dns://localhost:5053

Rainmaker · 11 Sep 2021 at 00:55

Ad_Augendae said:
Thanks I'll have a look at that if this starts to fail again.
I have add blocking on the router (Steven Blacks List) but I've had to fine tune some sites and I can't white list on the router, pi-hole is great as well but I just need too iron out a few little quirks. I'm still testing stuff and as yet not 100% settled on what combination to use security wise. I did read a few negative issues with https not being all it's cracked up to be, but is anything these days!

Try the OISD.nl list I posted earlier in the thread. It includes the Steven Black list and many more, but is curated to remove all the false positives and dead links. You shouldn't have to whitelist anything with OISD at all. It 'just works'.

As for what you may have read about DNS over HTTPS 'not being all it's cracked up to be', you have no concerns. TLS itself (the encryption used for HTTPS) is solid. The complaints come from corporate network admins, ISPs and governments etc, who can no longer control or see what people are doing, as DoH bypasses their logging/censorship etc. To us end users, that's not a bad thing, it's a bonus!

The main issue you're having with cloudflared timing out is easy enough to eliminate if you switch from PiHole+cloudflared and jump to AdGuard Home instead. Same idea, nicer UI, more features (including built in DoH, DoT, DoQ, dnscrypt - and upstream support for the same). Just add your upstream URLs, add the OISD list, point it to your domain's certificate, and away you go. Loads of fine tuning should you want it (DHCP server, per-client tagging and settings, service blocks and parental filtering etc). All free, very stable and mature. Have a look if you get frustrated by your issue and can't solve it.

Avathar77 · 11 Sep 2021 at 01:02

I have pi hole and as guard both(and guard on home assistant, pi hole in s separate pi4).
I have multiple vlans (main, guest, IoT and NoT).
Ad guard only seem to handle one vlan. Ss with pi hole I can't setup and guard to handle all of my vlans.

Ad_Augendae · 12 Sep 2021 at 22:12

Rainmaker said:
Try the OISD.nl list I posted earlier in the thread. It includes the Steven Black list and many more, but is curated to remove all the false positives and dead links. You shouldn't have to whitelist anything with OISD at all. It 'just works'.

As for what you may have read about DNS over HTTPS 'not being all it's cracked up to be', you have no concerns. TLS itself (the encryption used for HTTPS) is solid. The complaints come from corporate network admins, ISPs and governments etc, who can no longer control or see what people are doing, as DoH bypasses their logging/censorship etc. To us end users, that's not a bad thing, it's a bonus!

The main issue you're having with cloudflared timing out is easy enough to eliminate if you switch from PiHole+cloudflared and jump to AdGuard Home instead. Same idea, nicer UI, more features (including built in DoH, DoT, DoQ, dnscrypt - and upstream support for the same). Just add your upstream URLs, add the OISD list, point it to your domain's certificate, and away you go. Loads of fine tuning should you want it (DHCP server, per-client tagging and settings, service blocks and parental filtering etc). All free, very stable and mature. Have a look if you get frustrated by your issue and can't solve it.

Thanks for the tips on adguard etc, I have had a serious look over the last few days, first I built it on an old openwrt router which was way to slow then onto openwrt on and old pie that wasn't a great experience so I tried another rpi B+ which again was to slow. I will backup my rpi4 tomorrow and test again. But just for the moment I will leave you with a little giggle below which is RPI4 raspberrypi 'Cloudflared' host Vs RPI B+ adguard which didn't do too bad!

Code:

Final benchmark results, sorted by nameserver performance:
 (average cached name retrieval speed, fastest to slowest)

  192.168.  0.  8 |  Min  |  Avg  |  Max  |Std.Dev|Reliab%|
  ----------------+-------+-------+-------+-------+-------+
  - Cached Name   | 0.000 | 0.001 | 0.003 | 0.000 | 100.0 |
  - Uncached Name | 0.019 | 0.079 | 0.276 | 0.080 | 100.0 |
  - DotCom Lookup | 0.024 | 0.032 | 0.039 | 0.004 | 100.0 |
  ---<-------->---+-------+-------+-------+-------+-------+
                       raspberrypi
                Local Network Nameserver


  192.168.  0. 10 |  Min  |  Avg  |  Max  |Std.Dev|Reliab%|
  ----------------+-------+-------+-------+-------+-------+
  - Cached Name   | 0.003 | 0.007 | 0.012 | 0.003 |  97.8 |
  - Uncached Name | 0.024 | 0.074 | 0.267 | 0.067 |  97.8 |
  - DotCom Lookup | 0.033 | 0.042 | 0.053 | 0.004 |  97.7 |
  ---<-------->---+-------+-------+-------+-------+-------+
                         adguard
                Local Network Nameserver

Ad_Augendae · 14 Sep 2021 at 21:54

Anybody got a good working Unbound config for use with pihole+Unbound, It's been a while since I messed with unbound and I am unsure what all the funky extras do now. I'm using the official pi-hole one atm but would appreciate a push.

The short config
https://docs.pi-hole.net/guides/dns/unbound/

The longer jobbie
https://blacklab.net/set-up-pi-hole-as-truly-self-contained-dns-resolver/

Ad_Augendae · 14 Sep 2021 at 22:11

Maybe this one!
https://github.com/anudeepND/pihole-unbound

Ad_Augendae · 19 Sep 2021 at 19:54

I settled on an rpi 4 and rpi B+ as secondary for Pihole+Unbound using the config above with a few changes only to the rpi 4 config as follows. rpi4 being 192.168.0.8 I am quite happy with it so far.

Code:

 # Use 0x20-encoded random bits in the query to foil spoof attempts.
    # http://tools.ietf.org/html/draft-vixie-dnsext-dns0x20-00
    # While upper and lower case letters are allowed in domain names, no significance
    # is attached to the case. That is, two names with the same spelling but
    # different case are to be treated as if identical. This means dnswatch.com is the
    # same as DNSwatch.COM which is the same as DNSWATCH.COM.
    use-caps-for-id: yes

Code:

# One thread should be sufficient, can be increased on beefy machines. In reality for
    # most users running on small networks or on a single machine, it should be unnecessary
    # to seek performance enhancement by increasing num-threads above 1.
    num-threads: 4

jellybeard999 · 23 Sep 2021 at 19:01

Hi everyone. Hopefully someone can help.

I've pihole setup with DHCP, which seems to be working fine, however, when I change the DNS server on my router to the pihole IP I just get "ERR_NAME_NOT_RESOLVED" message, which seems to suggest the DNS isn't working

Any ideas?

I have a VM with pihole as my primary DNS and an original RPI as a backup as my secondary - both setup identically, though with a different DHCP range, and both seem to have the same issue.

Ad_Augendae · 23 Sep 2021 at 19:12

Can you not go into the setup and turn dhcp off for the moment on one of your pihole dns then set manual dns on your machine and point it to the pihole with dhcp disabled while still getting DHCP from your router. Also does Virgin dish out IPV6 address as well. I am on Virgin but I use it only in Modem mode.

jellybeard999 · 23 Sep 2021 at 19:16

Ad_Augendae said:
Can you not go into the setup and turn dhcp off for the moment on one of your pihole dns then set manual dns on your machine and point it to the pihole with dhcp disabled while still getting DHCP from your router. Also does Virgin dish out IPV6 address as well. I am on Virgin but I use it only in Modem mode.

I only just set up DHCP, as I read it can resolve DNS issues. It wasn't working before, when the router was handling DHCP.

Im on TalkTalk fibre with the TalkTalk hub.

Ad_Augendae · 23 Sep 2021 at 19:21

Ahh ok, I don't use dhcp from a pihole, maybe someone will be along more able to help and understand the caveats of said router.

jellybeard999 · 23 Sep 2021 at 19:26

Currently Ive left pihole handling DHCP, but the DNS on the router is set to 8.8.8.8.

I can see traffic going through the pihole, so it seems to be working, just this DNS issue

Ad_Augendae · 23 Sep 2021 at 19:27

DHCP doesn't resolve dns issues, if your router is falling over you will loose connection whilw everything gets it's shirt back on.

jellybeard999 · 23 Sep 2021 at 19:28

Ad_Augendae said:
DHCP doesn't resolve dns issues, if your router is falling over you will loose connection whilw everything gets it's shirt back on.

I'll change that back then

Ad_Augendae · 23 Sep 2021 at 19:28

can you change the DNS on that router to somthing else ?

jellybeard999 · 23 Sep 2021 at 19:30

Yes. It works fine with a public DNS (eg google 8.8.8.8). It doesn't with the Pihole IP.