Slow DNS resolution on pfsense

I'm using the default config on pfsense shown below:

server:
  forward-zone:
    name: "."
    forward-tls-upstream: yes
forward-addr: 45.90.28.0#2d52c4.dns.nextdns.io
    forward-addr: 2a07:a8c0::#2d52c4.dns.nextdns.io
forward-addr: 45.90.30.0#2d52c4.dns.nextdns.io
    forward-addr: 2a07:a8c1::#2d52c4.dns.nextdns.io

The stats in my pfsense resolver show that 45.90.30.0 is about 5x slower than 45.90.28.0.

I often get long (2-5 second) delays on lookups. I tried restarting the resolver but shortly after I got a 5000ms lookup to an uncached domain.