Question

I have a problem..
I want to download to XML some info from website ( this I know how to do ), i know how to dl info from website but problem is when first
i need to log in.

Its part of code:

 <form method="post" action="logowanie.php">
  <table class="center">
   <tr>
    <td><label for="username">Login:</label></td>
    <td><input type="text" name="username" value=""></td>
    <td rowspan="2"><div class="submit"><input type="submit" value="OK" name="submit"></div></td>
   </tr>
   <tr>
    <td><label for="password">Hasło:</label></td>
    <td><input type="password" name="password" value=""></td>

I need to some how enter username password and press button submit.
Any ideas , code samples how to do it ?
Thanks very much.

Im doing it in Java/Scala.

Answer 1

While the Apache HttpClient would work, using it from Scala is unidiomatic and verbose, and the Dispatch library provides a nice Scala wrapper that lets you write much more concise code. You could use it like this:

import dispatch._

val u = url("https://site.com/logowanie.php")
val info = Seq("username" -> "me", "password" -> "secret")

val client = new Http
client(u << info >>> System.out)

This example would simply print the response, but you can easily provide more sophisticated ways of handling the response, and your client now holds the cookies you need to continue interacting with the site.

Here’s a more complex example that I’ve used to log in to a system that required me to pick up a generated identifier from the form before logging in:

import dispatch._
import dispatch.jsoup.JSoupHttp._

val u = url("https://myuni.edu/something/login")
val info = Seq(
  "username"  -> "me",
  "password"  -> "secret",
  "warn"      -> "true",
  "submit"    -> "LOGIN",
  "execution" -> "e1s1",
  "_eventId"  -> "submit"
)

val client = new Http
val id = client(u </> (_.select("input[name=lt]").first.attr("value")))
client(u << info :+ ("lt" -> id) >>> System.out)

Here I’m using Dispatch’s JSoup support to parse the form page and pull out the identifier I need, which I can then add to the form parameters when I submit the request.