Question

So I made this basic chat application that is heavily reliant on security and encryption. I am running into a few problems in making everything work together.

I already managed to establish a connection between the client and server but I also want to have them share a common password which they will use to decrypt the whole chat session so only those two will view it. Also, the password is never transmitted over the wire for added security. This is where my problem starts.

As soon as my client and server connect, the server sends a user-specified challenge which is basically a string which the client will need to decrypt and send back, which then the server sees if it matches, and if it does communication will continue, much like WPA encryption in WiFi.

My encryption setup is AES, where I have a string and a key, then the whole thing is encrypted and decoded into Base64, then sent. The process is reversed obviously when decrypting. The server manages to send everything normally but the client returns null when decrypting even though it properly receives the encrypted string. I basically have one class that acts as a client or server depending on the user’s choice to become so.

Heres some code / stack traces:
Server Output:KCcOYuXTUD6SaXJQ4qIn7Q==
helloworld

Client Output:KCcOYuXTUD6SaXJQ4qIn7Q==
null

Encryption/Decryption Methods: http://pastebin.com/TtErYyz6

Socket Writing/Reading Method: http://pastebin.com/73QcQxva

Stacktrace on client side: http://pastebin.com/NdexZdyV
Note: When debugging line by line it just some how stops on like 27. in the socket pastebin link

Hope I have included enough detail to get a solution 😛
Thanks for any help, I have been working pretty hard on this project and even though some features are unnecessary, I will still try to implement them where applicable to learn as much as possible for when I start my programming degree next month

Complete code: http://pastebin.com/xkunfX37

Answer 1

I found a few issues in the host verification code (verifyHost function).

First, String comparisons are tricky:

if(tempChallenge == decryptedChallenge){

You’re comparing strings with ==, this will only work if they are the same string object. Proper string comparison is done with the String.equals function:

if( decryptedChallenge.equals(tempChallenge) ){

Second, input streams are also tricky. You’re reading everything from the input stream, and then attempting to read more. The server is sending one line to the client in challenge, and expects one back. While the client reads 3 lines.

Third, CPU eating loop in the server code. It will use up a lot of CPU, and calling in.readline() will block and wait for input any way.

while(!in.ready()){} // you can drop this line.

Here’s the pastebin with the modified code: http://pastebin.com/JxazrA0G