Question

I am trying to use AndroidHttpClient to download a CSV file. For some reason it fails in the line "HttpResponse response httpClient.execute(httpGet, localContext); and simply goes to "finally".

I’ve checked the URL in my browser – it is working fine.

I get no information from HttpResponse response – it simply like skips it.

I don’t get why. Any ideas?

Thanks
D

private ArrayList<String> retrieveStockFinParamsFromYahooApiUri(String yahooApiCall)
    {
        ArrayList<String> stockFinParamsFromYahooApiUriRows = new ArrayList<String>();
        String resultLine = "";

        BufferedReader reader = null;
        AndroidHttpClient httpClient = AndroidHttpClient.newInstance("yahooGetStockParams");
        HttpContext localContext = new BasicHttpContext();
        HttpGet httpGet = new HttpGet(yahooApiCall);

        try 
        {
            HttpResponse response = httpClient.execute(httpGet, localContext);              

            reader = new BufferedReader(new InputStreamReader(
                    response.getEntity().getContent()));

            while ((resultLine = reader.readLine()) != null) 
            {
                stockFinParamsFromYahooApiUriRows.add(resultLine);
            }
        }
        catch (IOException e) 
        {
            e.printStackTrace();
        } 
        finally 
        {
            if (reader != null) 
            {
                try 
                {
                    reader.close();
                } 
                catch (IOException e) 
                {
                    e.printStackTrace();
                }
            }
        }

        return stockFinParamsFromYahooApiUriRows;
    }

---

FURTHER INVESTIGATION (05.22.2012):

@Snicolas – thank you for your comments. After I read your comments I thought the problem might originate from the fact that I used the emulator and not my actual device to debug this. I thought the emulator might suffer some connection problems. When I tested it on my device – I noticed the problem still occurred – so this was NOT the problem.

So I change my code to catch (Throwable e) as you advised and got the following stack snapshot:

05-22 18:17:41.457: W/System.err(24552): android.os.NetworkOnMainThreadException
05-22 18:17:41.461: W/System.err(24552):    at android.os.StrictMode$AndroidBlockGuardPolicy.onNetwork(StrictMode.java:1099)
05-22 18:17:41.461: W/System.err(24552):    at java.net.InetAddress.lookupHostByName(InetAddress.java:391)
05-22 18:17:41.461: W/System.err(24552):    at java.net.InetAddress.getAllByNameImpl(InetAddress.java:242)
05-22 18:17:41.465: W/System.err(24552):    at java.net.InetAddress.getAllByName(InetAddress.java:220)
05-22 18:17:41.465: W/System.err(24552):    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:137)
05-22 18:17:41.465: W/System.err(24552):    at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:164)
05-22 18:17:41.468: W/System.err(24552):    at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:119)
05-22 18:17:41.468: W/System.err(24552):    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:360)
05-22 18:17:41.468: W/System.err(24552):    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:555)
05-22 18:17:41.472: W/System.err(24552):    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:487)
05-22 18:17:41.472: W/System.err(24552):    at android.net.http.AndroidHttpClient.execute(AndroidHttpClient.java:257)
05-22 18:17:41.472: W/System.err(24552):    at com.bewildering.app.StournamentDbAdapter$YahooStocksParams.retrieveStockFinParamsFromYahooApiUri(StournamentDbAdapter.java:1536)
05-22 18:17:41.476: W/System.err(24552):    at com.bewildering.app.StournamentDbAdapter$YahooStocksParams.run(StournamentDbAdapter.java:1709)
05-22 18:17:41.476: W/System.err(24552):    at com.bewildering.app.StournamentActivity.onCreate(StournamentActivity.java:65)
05-22 18:17:41.476: W/System.err(24552):    at android.app.Activity.performCreate(Activity.java:4465)
05-22 18:17:41.476: W/System.err(24552):    at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1049)
05-22 18:17:41.480: W/System.err(24552):    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1920)
05-22 18:17:41.480: W/System.err(24552):    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:1981)
05-22 18:17:41.480: W/System.err(24552):    at android.app.ActivityThread.access$600(ActivityThread.java:123)
05-22 18:17:41.484: W/System.err(24552):    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1147)
05-22 18:17:41.484: W/System.err(24552):    at android.os.Handler.dispatchMessage(Handler.java:99)
05-22 18:17:41.484: W/System.err(24552):    at android.os.Looper.loop(Looper.java:137)
05-22 18:17:41.484: W/System.err(24552):    at android.app.ActivityThread.main(ActivityThread.java:4424)
05-22 18:17:41.488: W/System.err(24552):    at java.lang.reflect.Method.invokeNative(Native Method)
05-22 18:17:41.488: W/System.err(24552):    at java.lang.reflect.Method.invoke(Method.java:511)
05-22 18:17:41.488: W/System.err(24552):    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:784)
05-22 18:17:41.492: W/System.err(24552):    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:551)
05-22 18:17:41.492: W/System.err(24552):    at dalvik.system.NativeStart.main(Native Method)

While looking for android.os.NetworkOnMainThreadExceptionI found the documentation which states that this is: The exception that is thrown when an application attempts to perform a networking operation on its main thread.

From that I conclude that I should put my call into an AsyncTask.

Can anyone please confirm that I understand this correctly – or am I missing something here? Strange that I hat to catch (Throwable e) in order to get to the bottom of this.

---

FURTHER INVESTIGATION (05.22.2012):

I can now confirm that this issue comes due to the fact that The exception… is thrown when an application attempts to perform a networking operation on its main thread. As I read this started with honeycomb.

Ths issue now: I get a 301 response from Yahoo (Moved Permanently). This is strange because when I cut paste the URL into the browser it works.

Any idea why this should work in the browser but not in the application? I must mention that this HTTP request receives a CSV file as a response. Could this be an issue?

---

FURTHER INVESTIGATION (05.23.2012):

For some reason when using (in my code) the Yahoo URI http://finance.yahoo.com/d/quotes.csv?s= the response is a 301 (Moved Permanently). When using http://download.finance.yahoo.com/d/quotes.csv?s= instead the response is 200 (OK). Even that both work fine in the browser. I was lucky enough to find this web page that gave me some clues regarding which URIs Yahoo reacts to. Now I can see the “reader” gets the data and everything is ALMOST fine. I still get some strange exception (still trying to figure it out): after reading all the lines in the response I catch a Throwable (I left it from my previous experiments) and later on in the stack I thus get 05-23 08:52:41.258: W/dalvikvm(933): threadid=11: thread exiting with uncaught exception (group=0x40a721f8) and something about an uncaught exception in doInBackground(Void... params). Still investigating…

Just solved this issue – it was just a mistake in calling the Yahoo API. Had to add &e=.csv to the end of the URI for it to work. Just like that: http://download.finance.yahoo.com/d/quotes.csv?s=KDHIX&f=sl1d1t1c1ohgv&e=.csv

---

FINAL CODE

private ArrayList<String> retrieveStockFinParamsFromYahooApiUri(String yahooApiCall)
    {
        ArrayList<String> stockFinParamsFromYahooApiUriRows = new ArrayList<String>();
        String resultLine = "";

        BufferedReader reader = null;
        AndroidHttpClient httpClient = AndroidHttpClient.newInstance("yahooGetStockParams");
        HttpContext localContext = new BasicHttpContext();
        HttpGet httpGet = new HttpGet(yahooApiCall);

        try 
        {
            HttpResponse response = httpClient.execute(httpGet, localContext);                  

            reader = new BufferedReader(new InputStreamReader(
                    response.getEntity().getContent()));

            while ((resultLine = reader.readLine()) != null) 
            {
                stockFinParamsFromYahooApiUriRows.add(resultLine);
            }

            if(response != null)
            {
                try
                {
                    response.getEntity().consumeContent();
                }
                catch (IOException e) 
                {
                    e.printStackTrace();
                }

            }
        }
        catch (IOException e) 
        {
            e.printStackTrace();                
        } 
        finally 
        {               
            if(httpClient != null)
            {
                httpClient.close();
            }

            if (reader != null) 
            {
                try 
                {
                    reader.close();
                } 
                catch (IOException e) 
                {
                    e.printStackTrace();
                }
            }
        }

        return stockFinParamsFromYahooApiUriRows;
    }

Answer 1

Chances are that you get an Exception in execute that is not an IOException. Then, your catch block never get executed and your finally block is executed before the method throws an exepected exception to the caller of retrieveStockFinParamsFromYahooApiUri.

You have at least 2 options :

• don’t catch an IOException but a prent class. Try Throwable, print the stack trace and see what happens. Then you can later add other catch blocks to handle the case. (Catching Throwables is a bad programming practice, but you can use temporarily).
• place the invocation of retrieveStockFinParamsFromYahooApiUri in a try catch block using the same mechanism as above to determine which exception classes to expect and act accordingly.

But you need to refine your understanding of the problem, catch everything possible, write the stack trace and check the logcat.

Here are some other advices :

• You need to check the result code of the request before reading its content.
• You could use Apache IOUtils to copy the content of the response input stream and cache it to a file.
• You could enable gzipping of the request and the response.
• You also need to close properly the HttpConnection in the finally block, you can consume the entity of the response to achieve this.