Ok, so what happened here? Well we migrated our two singlethreading BIND8 processes to one multithreading BIND9 process. The query-load did not change at all, but keeping its in-memory database in order using a modell that is thread-safe seems to have cost us a huge amount of user cpu time. Wasn't BIND9 supposed to even work better on an SMP system with two CPUs than two BIND8 processes for it only needs to keep one database in order? Well apparently not. Lets ask the bind9-workers Mailinglist i thought, and i got the following very helpful replies from Jinmei Tatuya:
I'd first like to recommend disabling threads. From my experiences, enable-threads buys almost nothing for most OSes, unfortunately. If you can allow the configuration with 2 named processes, it should provide better performance than a single BIND9 process with 2 threads. Secondly, according to your memory usage and configuration (i.e., max-cache-size=700M, 623m used), it looks like named reaches the high-water mark for the specified maximum, and tries to purge some cache entries. If this periodically happens, it may be the reason for the high CPU usage. So, if possible, it might help if you can add more memory which can afford the typical use case under the high-water level. ... If you go with disabling threads, you may also want to enable "internal memory allocation". (I hear that) it should use memory more efficiently (and can make the server faster) but is disabled by default due to response-performance reasons in the threaded case. You can enable this feature by adding the following line #define ISC_MEM_USE_INTERNAL_MALLOC 1 just before the following part of bind9/lib/isc/mem.c: #ifndef ISC_MEM_USE_INTERNAL_MALLOC #define ISC_MEM_USE_INTERNAL_MALLOC 0 #endif
I opted for the latter option, to disable threading, and applied this patch and set up my BIND9 configuration and loadbalancer to support two processes. You can achieve this by taking care never to bind more than one BIND9 process to the same IP-Address, giving it a custom pidfile and last but not least configure rndc to connect for another port on localhost for the second process.
