Rrom C#, reading/writing over SOCKET to JAVA and having some concurrency/socket issue.
I am trying to implement a server client application where the server is Java and the Client is C#. And they communicate over TCP/IP and exchanging some binary data between them.
Particularly I have a Packet class defined both in Java and C#. It has a header, key and value. Both Java and C# writes and reads the Packet to Socket in exact same way. This way I am able to send a request Packet from C#, process it on Java Server and send the response back as a Packet.
The original problem is way more complicated but I was able to boil it down to this “Simple” version.
I have implemented both server and client as described below. The code is also available at the bottom.
For me to state the problem you have to continue to read:)
Server(Java) side
On Server side I have a very dummy ServerSocket usage. It reads the incoming Packets and sends back almost the same Packet as response.
Client(C#) side
The client is a bit complicated. Client starts N(configurable) number of threads(I’ll call them user threads). One In Thread and One Out Thread. All user threads create a Call object with dummy request Packet and unique id. Then adds the call into a local BlockingCollection.
The Out Thread continuously reads the local BlockingCollection and sends all request packets to server
The In Thread also continuously reads response Packets from server and matches them to the Call objects(remember the unique call id).
If there is no response for a particular Call object within 5 sec intervals the user thread will complain about it by printing into Console.
Also there is a timer with 10 sec interval that prints how many transactions have been executed per second.
If you reached so far, thank you:).
Now the problem:
The code below, which is the implementation of what I described above works fine with Mono on Mac. On Windows it also doesn’t fail immediately with low number(<10) of user threads. As I increase the number of threads suddenly, somehow the response packets that client receives are getting corrupted. In terms of this application all user threads get stuck because the answer to their request is not received.
The question is why they are corrupted? As you see the threads that touche socket are In and Out threads. But somehow the amount of user threads affects the client and brakes it.
It looks like some concurrency or socket problem, but I could find it.
I have put the code for Server(Java) and Client(C#). They don’t have any dependency just compiling and running the Main methods on both(first server) shows the problem.
I appreciate if you read so far.
Server Code
import java.io.*;
import java.net.*;
import java.nio.ByteBuffer;
public class DummyServer {
public static void main(String[] args) throws IOException {
ServerSocket server = new ServerSocket(9900);
System.out.println("Server started");
for(;;){
final Socket socket = server.accept();
System.out.println("Accepting a connection");
new Thread(new Runnable(){
public void run() {
try {
System.out.println("Thread started to handle the connection");
DataInputStream dis = new DataInputStream(socket.getInputStream());
DataOutputStream dos = new DataOutputStream(socket.getOutputStream());
for(int i=0; ; i++){
Packet packet = new Packet();
packet.readFrom(dis);
packet.key = null;
packet.value = new byte[1000];
packet.writeTo(dos);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}).start();
}
}
public static class Packet {
byte[] key;
byte[] value;
long callId = -1;
private int valueHash = -1;
public void writeTo(DataOutputStream outputStream) throws IOException {
final ByteBuffer writeHeaderBuffer = ByteBuffer.allocate(1 << 10); // 1k
writeHeaderBuffer.clear();
writeHeaderBuffer.position(12);
writeHeaderBuffer.putLong(callId);
writeHeaderBuffer.putInt(valueHash);
int size = writeHeaderBuffer.position();
int headerSize = size - 12;
writeHeaderBuffer.position(0);
writeHeaderBuffer.putInt(headerSize);
writeHeaderBuffer.putInt((key == null) ? 0 : key.length);
writeHeaderBuffer.putInt((value == null) ? 0 : value.length);
outputStream.write(writeHeaderBuffer.array(), 0, size);
if (key != null)outputStream.write(key);
if (value != null)outputStream.write(value);
}
public void readFrom(DataInputStream dis) throws IOException {
final ByteBuffer readHeaderBuffer = ByteBuffer.allocate(1 << 10);
final int headerSize = dis.readInt();
int keySize = dis.readInt();
int valueSize = dis.readInt();
readHeaderBuffer.clear();
readHeaderBuffer.limit(headerSize);
dis.readFully(readHeaderBuffer.array(), 0, headerSize);
this.callId = readHeaderBuffer.getLong();
valueHash = readHeaderBuffer.getInt();
key = new byte[keySize];
dis.readFully(key);
value = new byte[valueSize];
dis.readFully(value);
}
}
}
C# Client code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net.Sockets;
using System.Net;
using System.IO;
using System.Collections.Concurrent;
using System.Threading;
namespace Client
{
public class Program
{
readonly ConcurrentDictionary<long, Call> calls = new ConcurrentDictionary<long, Call>();
readonly BlockingCollection<Call> outThreadQueue = new BlockingCollection<Call>(1000);
readonly TcpClient tcpClient = new TcpClient("localhost", 9900);
readonly private int THREAD_COUNT;
static int ops;
public static void Main(string[] args) {
new Program(args.Length > 0 ? int.Parse(args[0]) : 100).Start();
}
public Program(int threadCount) {
this.THREAD_COUNT = threadCount;
new Thread(new ThreadStart(this.InThreadRun)).Start();//start the InThread
new Thread(new ThreadStart(this.OutThreadRun)).Start();//start the OutThread
}
public void Start(){
for (int i = 0; i < THREAD_COUNT; i++)
new Thread(new ThreadStart(this.Call)).Start();
Console.WriteLine(THREAD_COUNT + " User Threads started to perform server call");
System.Timers.Timer aTimer = new System.Timers.Timer(10000);
aTimer.Elapsed += new System.Timers.ElapsedEventHandler(this.Stats);
aTimer.Enabled = true;
}
public void Stats(object source, System.Timers.ElapsedEventArgs e){
Console.WriteLine("Ops per second: " + Interlocked.Exchange(ref ops, 0) / 10);
}
public void Call() {
for (; ;){
Call call = new Call(new Packet());
call.request.key = new byte[10];
call.request.value = new byte[1000];
outThreadQueue.Add(call);
Packet result = null;
for (int i = 1;result==null ; i++){
result = call.getResult(5000);
if(result==null) Console.WriteLine("Call" + call.id + " didn't get answer within "+ 5000*i/1000 + " seconds");
}
Interlocked.Increment(ref ops);
}
}
public void InThreadRun(){
for (; ; ){
Packet packet = new Packet();
packet.Read(tcpClient.GetStream());
Call call;
if (calls.TryGetValue(packet.callId, out call))
call.inbQ.Add(packet);
else
Console.WriteLine("Unkown call result: " + packet.callId);
}
}
public void OutThreadRun() {
for (; ; ){
Call call = outThreadQueue.Take();
calls.TryAdd(call.id, call);
Packet packet = call.request;
if (packet != null) packet.write(tcpClient.GetStream());
}
}
}
public class Call
{
readonly public long id;
readonly public Packet request;
static long callIdGen = 0;
readonly public BlockingCollection<Packet> inbQ = new BlockingCollection<Packet>(1);
public Call(Packet request)
{
this.id = incrementCallId();
this.request = request;
this.request.callId = id;
}
public Packet getResult(int timeout)
{
Packet response = null;
inbQ.TryTake(out response, timeout);
return response;
}
private static long incrementCallId()
{
long initialValue, computedValue;
do
{
initialValue = callIdGen;
computedValue = initialValue + 1;
} while (initialValue != Interlocked.CompareExchange(ref callIdGen, computedValue, initialValue));
return computedValue;
}
}
public class Packet
{
public byte[] key;
public byte[] value;
public long callId = 0;
public void write(Stream stream)
{
MemoryStream header = new MemoryStream();
using (BinaryWriter writer = new BinaryWriter(header))
{
writer.Write(System.Net.IPAddress.HostToNetworkOrder((long)callId));
writer.Write(System.Net.IPAddress.HostToNetworkOrder((int)-1));
}
byte[] headerInBytes = header.ToArray();
MemoryStream body = new MemoryStream();
using (BinaryWriter writer = new BinaryWriter(body))
{
writer.Write(System.Net.IPAddress.HostToNetworkOrder(headerInBytes.Length));
writer.Write(System.Net.IPAddress.HostToNetworkOrder(key == null ? 0 : key.Length));
writer.Write(System.Net.IPAddress.HostToNetworkOrder(value == null ? 0 : value.Length));
writer.Write(headerInBytes);
if (key != null) writer.Write(key);
if (value != null) writer.Write(value);
byte[] packetInBytes = body.ToArray();
stream.Write(packetInBytes, 0, packetInBytes.Length);
}
}
public void Read(Stream stream)
{
BinaryReader reader = new BinaryReader(stream);
int headerSize = IPAddress.NetworkToHostOrder(reader.ReadInt32());
int keySize = IPAddress.NetworkToHostOrder(reader.ReadInt32());
int valueSize = IPAddress.NetworkToHostOrder(reader.ReadInt32());
this.callId = IPAddress.NetworkToHostOrder(reader.ReadInt64());
int valuePartitionHash = IPAddress.NetworkToHostOrder(reader.ReadInt32());
this.key = new byte[keySize];
this.value = new byte[valueSize];
if (keySize > 0) reader.Read(this.key, 0, keySize);
if (valueSize > 0) reader.Read(this.value, 0, valueSize);
}
}
}
This is a pretty common mistake: any
Readcall on a socket may not actually read as many bytes as you ask for, if they are not currently available.Readwill return the number of bytes read by each call. If you expect to read n bytes of data, then you need to call read multiple times until the number of bytes read adds up to n.