This link details on the various http client java implementations. I am looking for any links which could provide information about their NTLM protocol implementation differences.
On one of the windows machine, I found that the commons-http client 3.1 implementation fails with an authorization error (http status code 401) but the java 1.5 implementation succeeds. Since the java 1.5 implementation of the NTLM authentication protocol is not open source, I cannot compare the two implementations to understand what could be going wrong.
Update 1
I am aware of the fact that commons http client does not support NTLM v2. This link provides a comparison between various java http client implementations and mentions that apache http client provides a partial implementation of NTLM protocol. It does not detail more about it.
On troubleshooting the issue further I also found that the NTLM implementation provided by this link in combination with HTTPClient works on the windows machine (the commons http client implementation does not work as I mention above).
Update 2
By sniffing packets (using wireshack) I realized that the commons http client 3.1 ntlm protocol implementation does not generate the NTLM Response in the Type 3 message. This is generated by the JDK implementation. Do you know of any server/client setting which indicates that the authentication would fail if the NTLM response data is empty? (since the authentication failure we are facing is reproducible only on one machine. The authentication succeeds else where.)
We found the root cause of this issue. The configuration setting which lead to the authentication error was controlled by a security policy called NoLMHashPolicy. Enabling this policy means the windows server would no more store the LM Hash value for any password and it would use the NT Response hash to do the authentication. Since the NTLM protocol implementation from commons http client 3.1 library does not at all calculate the NT response one could face this error when this setting is enabled. More details about this setting can be found here.
As a solution one could just add an implementation of the AuthScheme interface and extract out the code from higher versions of the commons http client library (for e.g. 4.1.2) which computes the NT Response in the Type 3 message. Do not forget to update the length and the offset values for the NT Response fields.
Once the implementation of AuthScheme interface is ready it can be injected using the AuthPolicy.registeryScheme() method.