For what it's worth, one can reduce the impact of such issues by populating /etc/resolv.conf with "options timeout {t} attempts {a}" {t} being the time you find acceptable to wait for a dns response and {a} being the number of attempts. In a datacenter I also ensure that all occurrences of "domain" and "search" are removed from /etc/resolv.conf as they amplify the timeouts. Always use FQDN's in applications without exception. One can also populate /etc/gai.conf with parameters to prefer ipv4 over ipv6 or even disable ipv6 if your outbound servers talk through an edge device that will translate/proxy to ipv6 when required as having resolvers try both A and AAAA will amplify problems.<p>To see what I am talking about, create a ram disk on your edge recursive servers and configure your DNS daemon to write query logs to a file in that ramdisk. This will slow down your DNS servers a little but unless one sees the bigger picture of what is happening, one can not make educated decisions to retrofit the platform architecture. This is only require temporarily to analyze the behavior of the application and OS DNS requests.<p>Another useful improvement is to literally run Unbound on every single node and configure it to use multiple edge recursive servers and set the parameters to keep probing all of them but prefer the fastest ones. This reduces load on the edge recursive servers, improves application response times and minimizes DNS outage issues. Tune cache-min-ttl to what is optimal for your datacenter. I also find it useful to set delay-close higher on all the edge recursive servers to further reduce latency.<p>There are thousands of other improvements one could make but then this turns into a book.