Question

I have a page that comes back as an UnexpectedPage in HtmlUnit, the response is JSON. Can I use HTMLUnit to parse this or will I need an additional library?

Answer 1

HtmlUnit doesn’t support it. It can at highest execute a JS function. You need to check beforehand if the Content-Type of the returned response matches application/json and then use the suitable tool to parse it. Google Gson is useful in this.

WebClient client = new WebClient();
Page page = client.getPage("https://stackoverflow.com/users/flair/97901.json");
WebResponse response = page.getWebResponse();
if (response.getContentType().equals("application/json")) {
    String json = response.getContentAsString();
    Map<String, String> map = new Gson().fromJson(json, new TypeToken<Map<String, String>>() {}.getType());
    System.out.println(map.get("displayName")); // Benju
}

If the JSON structure is known beforehand, you can even use Gson to convert it to a fullworthy Javabean. You can find an example in this answer.