2

DNS outage - i/o timeout

This morning at approximately 10:15 UTC, my internal NextDNS resolvers started getting repeated i/o timeout messages when resolving on my internal network. For example:

Jun 01 08:14:14 resolver nextdns[434]: Query INTERNALIP UDP A pool.ntp.org. none (qry=30/res=30) 5001ms UDP: dns resolve: read: read udp INTERNALIP:60270->45.90.28.0:53: i/o timeout
Jun 01 08:14:14 resolver nextdns[434]: Query INTERNALIP UDP A 2.amazon.pool.ntp.org. none (qry=39/res=39) 5001ms UDP: dns resolve: read: read udp INTERNALIP:45917->45.90.28.0:53: i/o timeout
Jun 01 08:14:14 resolver nextdns[434]: Query INTERNALIP UDP A www.apple.com. none (qry=31/res=31) 5003ms UDP: dns resolve: read: read udp INTERNALIP:43859->45.90.28.0:53: i/o timeout
Jun 01 08:14:14 resolver nextdns[434]: Query INTERNALIP UDP AAAA skydrive.wns.windows.com. none (qry=42/res=42) 5018ms UDP: dns resolve: read: read udp INTERNALIP:50499->45.90.28.0:53: i/o timeout
Jun 01 08:14:14 resolver nextdns[434]: Query INTERNALIP UDP A www.apple.com. none (qry=31/res=31) 5015ms UDP: dns resolve: read: read udp INTERNALIP:52355->45.90.28.0:53: i/o timeout
Jun 01 08:14:14 resolver nextdns[434]: Query INTERNALIP UDP A www.apple.com. none (qry=31/res=31) 5021ms UDP: dns resolve: read: read udp INTERNALIP:36905->45.90.28.0:53: i/o timeout
Jun 01 08:14:14 resolver nextdns[434]: Query INTERNALIP UDP 65 ocsp2.apple.com. none (qry=33/res=33) 5002ms UDP: dns resolve: read: read udp INTERNALIP:33239->45.90.28.0:53: i/o timeout

Attempting a direct, manual resolution against the DNS resolvers also responds with a timeout:

C:\Users\user>nslookup google.com 45.90.28.215
DNS request timed out.
    timeout was 2 seconds.
Server:  UnKnown
Address:  45.90.28.215
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Request to UnKnown timed-out
C:\Users\user>nslookup google.com 45.90.30.215
DNS request timed out.
    timeout was 2 seconds.
Server:  UnKnown
Address:  45.90.30.215
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Request to UnKnown timed-out

3 replies

null
    • JohnathanKarate
    • 3 mths ago
    • Reported - view

    @nextdns

    • JohnathanKarate
    • 3 mths ago
    • Reported - view

    Additionally, the setup page for my profiles shows an issue with checking whether I'm using the config:

      • JohnathanKarate
      • 3 mths ago
      • Reported - view

      I have confirmed traffic leaves the WAN interface on my router; however no responses are received:

      user@router:~# tcpdump -i eth0 host 45.90.28.215 -U -n
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth8, link-type EN10MB (Ethernet), snapshot length 262144 bytes
16:17:03.158848 IP EXTERNAL_IP.55794 > 45.90.28.215.53: 1+ PTR? 215.28.90.45.in-addr.arpa. (43)
16:17:05.167723 IP EXTERNAL_IP.56073 > 45.90.28.215.53: 2+ A? google.com. (28)
16:17:07.174418 IP EXTERNAL_IP.64766 > 45.90.28.215.53: 3+ AAAA? google.com. (28)
16:17:09.182217 IP EXTERNAL_IP.64554 > 45.90.28.215.53: 4+ A? google.com. (28)
16:17:11.184687 IP EXTERNAL_IP.49255 > 45.90.28.215.53: 5+ AAAA? google.com. (28)

      I can also see the failures to access in the NextDNS container:

      WARN:  23:16:56 Endpoint failed: https://dns2.nextdns.io#45.90.30.0,2a07:a8c1::: roundtrip: context deadline exceeded
WARN:  23:16:51 Endpoint failed: https://dns1.nextdns.io#45.90.28.0,2a07:a8c0::: roundtrip: context deadline exceeded
WARN:  23:16:46 Endpoint provider failed: SourceHTTPSSVCProvider(dns.nextdns.io, https://dns.nextdns.io#45.90.28.0,2a07:a8c0::,45.90.30.0,2a07:a8c1::): exchange: roundtrip: dial tcp 45.90.28.0:443: connect: connection timed out

      Packet capture for the DoH address similarly is met with no response:

      user@router:~# tcpdump -i eth0 host 45.90.28.0 -U -n
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth8, link-type EN10MB (Ethernet), snapshot length 262144 bytes
16:20:37.463466 IP EXTERNAL_IP.42193 > 45.90.28.0.443: Flags [S], seq 529138403, win 29200, options [mss 1460,sackOK,TS val 30586445 ecr 0,nop,wscale 7], length 0
16:20:37.463466 IP EXTERNAL_IP.42192 > 45.90.28.0.443: Flags [S], seq 2509644044, win 29200, options [mss 1460,sackOK,TS val 30586445 ecr 0,nop,wscale 7], length 0
16:20:38.466773 IP EXTERNAL_IP.42193 > 45.90.28.0.443: Flags [S], seq 529138403, win 29200, options [mss 1460,sackOK,TS val 30587448 ecr 0,nop,wscale 7], length 0
16:20:38.466778 IP EXTERNAL_IP.42192 > 45.90.28.0.443: Flags [S], seq 2509644044, win 29200, options [mss 1460,sackOK,TS val 30587448 ecr 0,nop,wscale 7], length 0
16:20:40.470478 IP EXTERNAL_IP.42193 > 45.90.28.0.443: Flags [S], seq 529138403, win 29200, options [mss 1460,sackOK,TS val 30589452 ecr 0,nop,wscale 7], length 0
16:20:40.470478 IP EXTERNAL_IP.42192 > 45.90.28.0.443: Flags [S], seq 2509644044, win 29200, options [mss 1460,sackOK,TS val 30589452 ecr 0,nop,wscale 7], length 0
16:20:44.475009 IP EXTERNAL_IP.42193 > 45.90.28.0.443: Flags [S], seq 529138403, win 29200, options [mss 1460,sackOK,TS val 30593456 ecr 0,nop,wscale 7], length 0
16:20:51.742053 IP EXTERNAL_IP.36346 > 45.90.28.0.443: Flags [S], seq 809331736, win 64240, options [mss 1460,sackOK,TS val 461333449 ecr 0,nop,wscale 7], length 0
16:20:52.490557 IP EXTERNAL_IP.42193 > 45.90.28.0.443: Flags [S], seq 529138403, win 29200, options [mss 1460,sackOK,TS val 30601472 ecr 0,nop,wscale 7], length 0
Login to reply

Content aside

  • 2 Likes
  • 3 mths agoLast active
  • 3Replies
  • 90Views
  • 2 Following