Home/ Questions/Q 7952243
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-04T02:48:04+00:00

I have a problem with our Jetty application server. Since yesterday, we have a

  • 0

I have a problem with our Jetty application server. Since yesterday, we have a big issue. Sometimes, Jetty seems to simply hang and don’t want to do any RPC call. If I restart it, it return to its normal state, but the problem will comeback few hours later.

I noticed that the log of Nginx prompt this error when we have the problem :

2012/05/17 17:00:47 [error] 14728#0: *506735 upstream timed out (110: Connection timed out) while reading response header from upstream, client: xx.xxx.xxx.xx, server: www.MY-SERVER.com, request: "POST /com.xxxx.xxxx.XXXX/Service HTTP/1.1", upstream: "http://127.0.0.1:8080/com.xxxx.xxxx.XXXX/Service", host: "www.MY-SERVER.com", referrer: "https://www.MY-SERVER.com/"

Even when it’s working well, Nginx prompt this warning :

2012/05/17 17:04:44 [warn] 14728#0: *506906 a client request body is buffered to a temporary file /var/cache/nginx/client_temp/0000088415,

client: xx.xxx.xxx.xx, server: http://www.MY-SERVER.com,
request: “POST /report HTTP/1.1”, host: “www.MY-SERVER.com”
Is it alarming?

I’ve heard that when Jetty hang like that it may be because of a thread issue. How can I verify that? Here is my Server Thread Pool configuration, may be it can help diagnostic the problem.

<!-- =========================================================== -->
    <!-- Server Thread Pool                                          -->
    <!-- =========================================================== -->
    <Set name="ThreadPool">
      <!-- Default queued blocking threadpool -->
      <New class="org.eclipse.jetty.util.thread.QueuedThreadPool">
        <Set name="minThreads">10</Set>
        <Set name="maxThreads">200</Set>
        <Set name="detailedDump">false</Set>
      </New>
    </Set>

    <!-- =========================================================== -->
    <!-- Set connectors                                              -->
    <!-- =========================================================== -->

    <Call name="addConnector">
      <Arg>
          <New class="org.eclipse.jetty.server.nio.SelectChannelConnector">
            <Set name="host"><Property name="jetty.host" /></Set>
            <Set name="port"><Property name="jetty.port" default="8080"/></Set>
            <Set name="maxIdleTime">300000</Set>
            <Set name="Acceptors">2</Set>
            <Set name="statsOn">false</Set>
            <Set name="confidentialPort">8443</Set>
            <Set name="lowResourcesConnections">20000</Set>
            <Set name="lowResourcesMaxIdleTime">5000</Set>
          </New>
      </Arg>
    </Call>
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    First things first you need to mention what jetty version you are dealing with, that is pretty important in comparing the situation against some known issues.

    In general I can only recommend that you update to the latest jetty release, either 7.6.3.v20120416 or 8.1.3.v20120416.

    Well that and you need to take a thread dump to see where things are stuck. While there have been a couple of issues stemming from our attempted fixes to nio bugs in the jvm, and some ssl issues related to half-closes being used far more often in newer browsers then previously…the majority of issues in server hangs are from the applications deployed in jetty itself. So taking a thread dump, or ideally several in a row will help isolate where your having issues.

    Another thing to check is what java version your using, the java 6 patch level 2x’s were almost all bad in one way or another with nio and ssl, only the latest 3x have restored some semblance of normality to things.

    So first you really need to take some thread dumps to get an idea of what is going on in your server before you look to see if its a jetty specific issue. If you can post some bits of the stack trace showing obvious jetty issues then your getting somewhere. I can’t count the number of times I have seen a situation like this only to take a thread dump and see its database contention or something similar.

    • 0