Question

I have started writing an application that gets in a certain website and gets the information in the pages in a nicer cleaner way. My first problem is getting authenticated so I could download the pages. This is the site: https://www.ims.tau.ac.il/Tal/.
It is in Hebrew but the top form goes (top to bottom): username, id number, password.

When I fill the form out and click the SUBMIT button I press F12 and see the POST message:

Request URL:https://www.ims.tau.ac.il/Tal/Login_Chk.aspx?rk=
Request Method:POST
Status Code:302 Found

Request Headersview parsed

POST /Tal/Login_Chk.aspx?rk= HTTP/1.1    
Host: www.ims.tau.ac.il    
Connection: keep-alive 
Content-Length: 82    
Cache-Control: max-age=0   
Origin: https://www.ims.tau.ac.il
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/535.2   
Content-Type: application/x-www-form-urlencoded    
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8    
Referer: https://www.ims.tau.ac.il/Tal/    
Accept-Encoding: gzip,deflate,sdch    
Accept-Language: en-US,en;q=0.8    
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3    
Cookie: ASP.NET_SessionId=2mi4i555jfibfb55wolrse55

Query String Parametersview URL encoded

rk:
Form Dataview URL encoded
user:MYUSERNAME
id_num:MYIDNUMBER
pass: MYREALPASSWORD
Enter.x:46
Enter.y:18
javatest:9
src:

Response Headersview parsed

HTTP/1.1 302 Found    
Date: Wed, 16 Nov 2011 21:25:46 GMT    
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET    
X-AspNet-Version: 2.0.50727    
Location: Sys/Main.aspx?id=021653167&sys=tal&rk=&dt=16/11/2011 11:25:46 PM    
Cache-Control: private    
Content-Type: text/html; charset=utf-8   
Content-Length: 207

I tried to do it by myself from pieces of code I found on other posts in the forum but I couldn’t get it to work:

public class MyTauRobot implements Runnable {

    private static final String TAG = "MyTauActivity";

    String name;
    String id;
    String pass;
    String result;

        public MyTauRobot() {
        super();
    }

    public MyTauRobot(String name, String id, String pass) {
        super();
        this.name = name;
        this.id = id;
        this.pass = pass;
    }

    @Override
    public void run() {
        HttpClient postClient = new DefaultHttpClient();
        HttpPost httpPost = new HttpPost("https://www.ims.tau.ac.il/Tal/Login_Chk.aspx?rk=");
        HttpResponse response;


        try {

            List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(2);
            nameValuePairs.add(new BasicNameValuePair("user", name));
            nameValuePairs.add(new BasicNameValuePair("id_num", id));
            nameValuePairs.add(new BasicNameValuePair("pass", pass));

            httpPost.setEntity(new UrlEncodedFormEntity(nameValuePairs));

            response = postClient.execute(httpPost);

            if(response.getStatusLine().getStatusCode() == 200) {
                HttpEntity entity = response.getEntity();

            }           

            Log.v(TAG,EntityUtils.toString(response.getEntity())); 

        } catch (Exception e) {
            Log.d(TAG, e.toString());
            Log.d(TAG, e.getCause().toString());
            }
        }
}

When I run it I get an exception.

org.apache.http.client.ClientProtocolException
11-19 22:42:48.758: D/MyTauActivity(726): org.apache.http.ProtocolException: Invalid redirect 
URI: Default.aspx?id=&rk=&src=&dt=20/11/2011 12:42:48 AM

UPDATE:
I still get an exception but I’ve found out that atlist it seems to be logging in right
the exception comes because the redirect is relative so i get “main.aspx?…”
instead of “http://www.tau.ac.il/Tal/Sys/main.aspx?…”

How do i fix it?

org.apache.http.ProtocolException: Invalid redirect URI: Sys/Main.aspx?id=021653167&sys=tal&rk=&dt=25/11/2011 8:50:56 AM

Answer 1

Well, I found the solution.

Problem was that the server was replying with relative redirects and the defaultHttpClient could not follow. This is weird since I set on the defaultHttpClient:

this.getParams().setParameter(ClientPNames.REJECT_RELATIVE_REDIRECT, false);

But since it didn’t get it to work I decided to try to make a relative address to an absolute address and I made it work.

All you have to do is:

1. Create a custom redirect handler and override the getLocation as follows:

   class CustomRedirectHandler extends DefaultRedirectHandler {
       @Override
       public URI getLocationURI(HttpResponse response, HttpContext context)
               throws ProtocolException {
           response.setHeader("Location", "http://www.ims.tau.ac.il/Tal/"+ response.getFirstHeader("Location").getValue().replace(" ", "%20"));        
           return super.getLocationURI(response, context);
       }
   }
   

where instead of “http://www.ims.tau.ac.il/Tal/” you write your own absolute base location for the files.

For example you need http://www.google.com/index.html but the server just redirects to index.html. You put in “http://www.google.com/” in the location header and concat the server location reply just like in the code above.

BTW the server was replying with spaces which again DefaultHttpClient had problem with so you replace spaces with %20 to get it to work.

1. Set your default/custom HttpClient to use the new redirect handler like this:

   httpClient.setRedirectHandler(new CustomRedirectHandler());