Question

I want to access a webpage programmatically and extract some information from it.

I want to log in to some website through Java code and make the server feel that the request is actually coming from a real browser.

I am almost there albeit one problem: the website requires a parameter - "sessid" to be passed with to be passed with every request which keeps on changing with every request.

For e.g when I first access the page the sessid=90334 and at the next page its like sessid=78204.

Therefore the url I pass should contain the value of sessid otherwise the authentication fails: www.somesite.com/somepage.php?sessid=75749.

The webpage contains one <input> tag which holds the value of sessid and i have to retrieve the value of that tag.

How can i do that? The tag is like this:

<input type="hidden" name="sessid" value="69529">

I am able to read the whole webpage successfully using the following code:

   BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));

    StringBuilder response = new StringBuilder();
    String line;
    while ((line = rd.readLine()) != null) {
        response.append(line);
    }

Answer 1

You can use indexOf method of StringBuilder class:

    String startInputFragment = "<input type=\"hidden\" name=\"sessid\" value=\"";
    int startIdx = response.indexOf(startInputFragment);
    if (startIdx >= 0) {
        int endIdx = response.indexOf("\">", startIdx);
        String val = response.substring(startIdx + startInputFragment.length(),
                endIdx);
        System.out.println("-->" + val + "<--");
    } else {
        //tag not found: you may throw an ex or do something else
    }