NextDNS not reachable over port 53

I have an intermittent problem.

The problem

I have nextDNS configured in my Unifi router as the DNS server for my Internet connection (IPv4 and IPv6 configured). This works for a while, until the router cannot resolve any address anymore. Running an nslookup from the command prompt on a client in the network, manually pointing to the NextDNS server also doesn't work.

After a while the connectivity comes back. Resolving via DoH doesn't seem to be affected (by the router has a bug here that results in not properly resolving internal addresses...)

This has been going on for some time now. I was able to recreate this from my work network. I created a new server, bound it to my work IP and it got me the same results.

What I tried

1. The NextDNS diagram script runs normally.
2. Disabled all firewall rules that affect DNS (no changes there in the pas 6 months; those rules worked well for a long time)
3. Pointing clients directly to NextDNS via DHCP
4. Removed the IPv6 DNS servers from the Internet config
5. Going to try removing the IPv4 servers from the Internet config next.