Question

I’m creating an Android app that should do the following;

1. Use a form on a https (SSL!) page to login and receive a cookie
2. Issue httpGET actions to get html
3. parse that html and show it in a view, list or something.

I’ve been fooling around with Jsoup, httpUnit and HTMLUnit for quite some time now, but I’m running in to several problems;

A. Login is fine, works.. (I get the website’s welcome page) but then, when I issue a GET statement (and include the cookie), I am redirected to the login form. So the response html is not what I expected. (might have something to do with a keepalivestrategy?)

B. InputBuffers are too small to receive entire HTML pages and set them up for parsing.

NB : I do not have control over the webserver

I’m totally new at this, so a tutorial or code snippets would be helpful.

For instance, this is what I use to login to the website :

public int checkLogin() throws Exception {

    ArrayList<NameValuePair> data = new ArrayList<NameValuePair>();
    data.add(new BasicNameValuePair("userid", getUsername()));
    data.add(new BasicNameValuePair("password", getPassword()));
    data.add(new BasicNameValuePair("submit_login", "Logmein"));

    Log.d(TAG, "Cookie name : " + getCookieName());
    Log.d(TAG, "Cookie cont : " + getCookie());

    HttpPost request = new HttpPost(BASE_URL);
request.getParams().setBooleanParameter(CoreProtocolPNames.USE_EXPECT_CONTINUE, false);
request.getParams().setParameter("http.protocol.handle-redirects",false);
    request.setEntity(new UrlEncodedFormEntity(data, "UTF-8"));

    HttpResponse response;

    httpsclient.getCookieStore().clear();

    List<Cookie> cookies = httpsclient.getCookieStore().getCookies();
    Log.d(TAG, "Number of Cookies pre-login : "  + cookies.size());

    response = httpsclient.execute(request);

    cookies = httpsclient.getCookieStore().getCookies();
    Log.d(TAG, "Number of Cookies post-login : "  + cookies.size());
    String html = "";

    // Problem : buffer is too small!

    InputStream in = response.getEntity().getContent();
    BufferedReader reader = new BufferedReader(new InputStreamReader(in));
    StringBuilder str = new StringBuilder();
    String line = null;
    while ((line = reader.readLine()) != null) {
        str.append(line);
    }
    in.close();
    html = str.toString();

    Document doc = Jsoup.parse(html);
    Log.v(TAG, "Ik heb nu dit : " + doc.toString());

    if (cookies.size() > 0){
        storeCookie(cookies.get(0).getName(), cookies.get(0).getValue());
        return MensaMobileActivity.REQUEST_SUCCESS;
    } else {
        return MensaMobileActivity.REQUEST_ERROR;           
    }

}

Answer 1

You don’t handle the SSL certificate at all, that’s at least a part of the problem. I struggled starting to learn this recently as well. This block of code will grab the SSL cert from the webpage you’re accessing.

    try {
        URL url = new URL(YOUR_WEBPAGE_HERE);
        HttpsURLConnection connect = (HttpsURLConnection)url.openConnection();
        connect.connect();
        Certificate[] certs = connect.getServerCertificates();

        if (certs.length > 0) {

            cert = new File("YOUR_PATH_TO_THE_FILE");
            //write the certificate obtained to the cert file.
            OutputStream os = new FileOutputStream(cert);
            os.write(certs[0].getEncoded());

            return true;
        }
    }
    catch (SSLPeerUnverifiedException e) {
        e.printStackTrace();
    }
    catch (MalformedURLException e) {
        e.printStackTrace();
    }
    catch (IOException e) {
        e.printStackTrace();
    }
    catch (CertificateEncodingException e) {
        e.printStackTrace();
    }