Question

When i do traceroute from one solaris m/c to another using 10 gig network interface it takes 0.073 ms for 40 bytes packets.

When i do the same thing in java, the time is way longer. It is longer even after 10K iteration. What could be the reason ?

Java: Sender (Snippet)

Socket sendingSocket = new Socket(address, RECEIVER_PORT);
 sendingSocket.setTcpNoDelay(true);
 OutputStream outputStream = sendingSocket.getOutputStream();
 byte[] msg = new byte[64]; // assume that it is populated. 
 for (int i = 0; i < 10000; i++) {
    long start = System.nanoTime();
    outputStream.write(msg,0,64);
    outputStream.flush();

    inputStream.read(msg,0,64);  // inputStream is initialized like outputstream 
    long end = System.nanoTime(); 
 }  

It takes way longer 69 millis and it does not even depends upon the byte size. Even if i reduce it to say 1 byte array, it still takes 69 millis. Any comment/Suggestion ?

Other Observation:
1. OutputStream.write and flush only takes 6 micros.
2. Similarly on the other end TCPReceiver side which receives and writes back, it only takes 6 micros.

Solution:
Thank you all you responded for this query.
I found that it was due to the socket buffer size:

Default buffer size set on solaris m/c.

Received Buffer Size 49152.

Sending Buffer Size 7552.

I increased the socket buffer size and the performance almost matches traceRoute.

Answer 1

You are not comparing like with like. ICMP and TCP are way different protocols.

If you want to decide if the latency is in your code, the JVM, Solaris TCP stack or the network you’ll have to start with tcpdump / wireshark etc.