Question

I’m working on an Android project and I’m currently playing around with JSoup to extract data from a website for the application.

The website I’m targeting is here.

And I want to extract the main information text. The xpath for that div is

//div[@id='wikiAbstract']

My complete code is as follows

public class Main extends Activity {
/** Called when the activity is first created. */
@Override
public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.main);

   TextView tv = (TextView) findViewById(R.id.textView1);
   Document doc = null;

   try {
       doc = Jsoup.connect("http://www.last.fm/music/Bright+Eyes").get();

   } catch (IOException e) {
       e.printStackTrace();
   }

   Element divs = doc.select("div#wikiAbstract").first();
   tv.setText(divs.text());

}

However, I’m getting a Null Pointer Exception. I have tested the same code on other websites and divs and it works perfectly. I can’t understand why this is different.

I would appreciate any help/feedback from anyone, thanks in advance.

Answer 1

Ah, ok so the html content from here was coming from the mobile version of the last.fm website so the div id’s were different.

For anyone who might encounter this, you can add a user-agent request header to the connection to ensure the full website is requested or just ensure you get the correct div id’s from the mobile site

Document doc = Jsoup.connect(myURL)
                .userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0.1) Gecko/20100101 Firefox/8.0.1")
                .get();