One of the hard things to troubleshoot in Windows domains is NTLM authentication and the interesting MaxConcurrentAPI setting. When user account to be authenticated on the server does not belong to the local account database, the server must forward the authentication to a domain controller. It does so over the Netlogon secure channel. As such, it is governed by the MaxConcurrentApi setting as to how many outgoing authentication requests are allowed at a time. When you have a loaded server, this can create a problem, an example of which is described in an ISA server configuration article. Finding the root cause is very laborious process, so we’ve added performance counters to Netlogon to allow troubleshooting this issue. For Windows 2003, a hotfix is required, but for Windows 2008 and later it is present in the core install.
I was thinking of writing a very thorough guide on how authentication works in this type of scenario, but found out that the ISA documentation team has beaten me to it. Their description is targeted to ISA server, but you can replace it with any other application server doing user authentication using NTLM.
There are also another two articles (1, 2) that are useful and helpful material to read related to the throughput of Netlogon secure channel traffic. If you still think there is more that can be done to describe this setting and issues surrounding authentication, feel free to drop me a line or leave a comment.
P.S. If you are hitting this issue, you should seriously audit your network and applications as to why they are not using Kerberos for authentication. There are a few common pitfals as to why NTLM ends up being the auth protocol, but that might go in another post of its own.