DNSFilters problematic test tool that misrepresents every DNS Service including NextDNS.

Hey
updated 2 yrs ago
9replies

DNSF_SecurityAudit_kT4jaAHarX chrome_av4to6h7ew

I've been a NextDNS user for years now but I keep my eye on other services just to see what/how they are doing to compare it to NextDNS as I honestly believe that it's the benchmark DNS.

So I was looking at DNSFilters website and they had a video comparing it to OpenDNS with their Lifesaver Program. Their DNSFilter Security Audit.

So I told myself why not test my current config on NextDNS I did the test and 0% on all categories Botnet/Malware/Phishing & Deception.

I said okay, why not add all the filters that are updated, 0%, now I was annoyed and honestly shocked, so ended up enabling all the parental control options since it had some domains that were not blocked that lead to those sites, 0%.

I had enough and by constantly flushing the DNS (I did this on every run of the test) then looking at the TDLs on analytics, I blocked every single TDL that DNSFilter was hitting and only allowed nextdns.io and dnsfilter.com since the app would hit their site and didn't want to hinder the apps connection.

So now I should see 100% the only domain that wasn't blocked was their own dnsfilter.com at worse if they also test IPs where NextDNS doesn't block them since issues with multiple sites having the same IP and that causing false positives etc from what I know. It should still get something decent not 0% that makes no sense.

Time to say okay, you know maybe I messed up on my testing, so going on to their own video, when testing against Quad9 (in my testing NextDNS was either on par or better with my lean OISD only test where I wasn't held back from using anything.)

They gave Quad9 0% for Botnet, 30% for Malware and a 0% for Phishing & Deception.

I don't know what they are doing wrong, but man, with every single domain blocked but their own giving that a 0% rating, that's crazy. I just wanted let everyone know so they can avoid testing with this tool as honestly it doesn't represent any sort of actual protection.

I've added two screenshots of separate tests done with a DNS Flush on Windows using a different config that only this laptop uses through NextDNSs app.

- Hey
- 2 yrs ago
- Reported - view
I know that DNSFIlter is a completely different product but since the tool is meant for testing other products against their service (DNSFilters). Seeing such weirdness where I don't even have any words.

I wanted to share it with the public. Going on my own testing journey and seeing others test DNS Services where NextDNS is either at the top or extremely close to the top and where I had on par or better results compared to Quad9 with NextDNS being setup in an extremely lean manner with OISD/Security and Privacy options turned on.

I was simply shocked to how they misrepresent other services and their abilities to block malicious content online, I just had to make this thread.
- Coral_River
  
  2 yrs ago
  
  Reported - view
  Hey How many things were blocked by OISD blocklist only in your security testing?
- Hey
  
  2 yrs ago
  
  Reported - view
  servilo From my testing it wasn't a lot but the test was for 0 day / Malicious Sites reported within few hours. So more or less before OISD even had the time to update to include them. But doing testing with known malicious sites, OISD does far better.
  
  To put it in a single sentence, for threats that are older than 5-6 hours or when OISD updated, it does a great job, and since there are other layers on NextDNS, that Zero Day aspect is covered as well.
  
  Especially AI Though since I've seen 60-70% detection purely by AI on those Zero day tests I did, I was truly amazed.
- Mikey
- 2 yrs ago
- Reported - view
hey first thank you for downloading our new Security Audit tool. I've done some testing based on your feedback and want to share the results.

I have two screenshots. The first is using NextDNS's DNS IPs and the second is the Agent. Both utilize the same policy which blocks everything available from NextDNS.

NextDNS DNS IPs

NextDNS Agent

You can see the results from the DNS IPs are quite good on threat efficacy, better than most solutions we've tested. NextDNS's agent uses VPN technology and must obscure the DNS response from our Security Audit tool. I'll conduct more tests on other providers to see if agent vs DNS IPs has similar results and we will make a best effort to update the tool or provide a disclaimer.

We at DNSFilter very much respect NextDNS's position in the market and the easy to use service. I would go so far as saying the respect is mutual, hi Romain and Olivier

It's also true that DNSFilter has unique threat intelligence that supports our claims of better threat efficacy.

Again, thank you for reviewing the landscape and keeping us honest.
- Hey
  
  2 yrs ago
  
  Reported - view
  Mikey - DNSFilter Thank you for your response and explaining the 0% issue as I was honestly more shocked than anything to see that, didn't know about the fact that there was obfuscation that was impacting the results. I'll try to contact support to hopefully have the thread removed since now I'm misrepresenting the tool because of something I didn't know. Hopefully I can make a follow up test based on IP with DNS Filter as well, although I've already used the trial once, so would also like to ask to use it again tomorrow for additional testing.
  
  Thanks for letting me know and hopefully I'll make a new thread about all this and get this thread removed.
- Coral_River
  
  2 yrs ago
  
  Reported - view
  Mikey - DNSFilter
  
  I'm not an IT specialist so I would like to ask some questions:
  1) What is the the difference between NextDNS DNS IPs and NextDNS Agent?
  
  2) Which security options were enabled for the first screen - NextDNS DNS IPs ?
  
  3) Can you please provide testing for Quad9?
- Hey
  
  2 yrs ago
  
  Reported - view
  Mikey - DNSFilter I've also just done a test with UDP / Putting the IP on the Router and Linking it on the NextDNS setup page, then had 6% Malware and 7% Phishing detection. So it's as you said the app not getting the results it wants on the client but still something seems off as it's not even close to that 80% mark on my end even though I've only added an additional filter and turned on additional beta security layers.
  
  On the DNSFilter page it puts Quad9 at 30% for Malware Detection while also putting Google, Cloudflare and Xfinity on the same 30%
  
  After the first response I said yeah I've made a huge mistake but while VPN aspect might not be accommodated on the app, there still seems to be something wrong with the apps way of doing things from the video.
  
  On every other test I've done and seen online NextDNS is usually better than Quad9, but when it comes to Google and Cloudflare they have far worse results, so the Quad9 results also don't add up.
  
  DNSF_SecurityAudit_dRhzRQXmPz
- Mikey
  
  2 yrs ago
  
  Reported - view
  hey, servilo; thank you for bringing that up. Quad9 is an interesting case because when a domain is blocked by that service it just returns NXDOMAIN (nonexistant) and the sets the authority flag to indicate blocked or actually nonexistant. See more in Quad9's support.
  
  Our tool does not take this into account for Quad9 and we are re-evaluating the validity of the tool based on your feedback.
  
  When testing DNS filtering solutions be sure to flush the DNS cache of your OS between tests.
  
  Windows 7+
  
  Press ⊞ Win + X
  
  Right-click Command Prompt and select Run as Administrator.
  
  Run the following command and hit enter: ipconfig /flushdns
  
  macOS 10.9+
  
  Launch the Terminal application
  
  Run the following command and hit enter:
  sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponder
- Hey
  
  2 yrs ago
  
  Reported - view
  Mikey - DNSFilter I cleared the cache on every run when doing multiple runs with IP linking through my router and got those scores.
  
  So, thanks for the response but with both NextDNS and Quad9 especially with Quad9 as it was mentioned on the main video it's not the greatest to see it as a comparison done by the company and on the DNSFilter site.
  
  Hopefully that video can be re-done to appropriately show results,