Question

I am fairly new to Java, at least regarding interacting with web. Anyway, I am making an app that has to grab HTML out of a webpage, and parse it.

By parsing I mean finding out what the element has in the ‘class=”” ‘ attribute, or in any attribute available in the element. Also finding out what is inside the element. This is where I have searched so far: http://www.java2s.com/Code/Java/Development-Class/HTMLDocumentElementIteratorExample.htm

I found very little regarding this.

I know there are lots of Java parsers out there. I have tried JTidy, and the default Swing parser. I would prefer to use the built-in-to-java parser.

Here is what i have so far (this is just method for testing how it works, proper code will come when i know what & how. Also connection is a URLConnection variable, and connection has been established before this method gets called. < just to clarify):

public void parse() {
        try {

            InputStream is = connection.getInputStream();
            InputStreamReader isr = new InputStreamReader(is);
            BufferedReader br = new BufferedReader(isr);

            String line;
            while ((line = br.readLine()) != null) {
                System.out.println(line);
            }

            // copied from http://www.java2s.com/Code/Java/Development-Class/HTMLDocumentElementIteratorExample.htm
            HTMLEditorKit htmlKit = new HTMLEditorKit();
            HTMLDocument htmlDoc = (HTMLDocument) htmlKit.createDefaultDocument();
            HTMLEditorKit.Parser parser = new ParserDelegator();
            HTMLEditorKit.ParserCallback callback = htmlDoc.getReader(0);
            parser.parse(br, callback, true);

            // Parse
            ElementIterator iterator = new ElementIterator(htmlDoc);
            Element element;

            while ((element = iterator.next()) != null) {
                AttributeSet attributes = element.getAttributes();

                Object name = attributes.getAttribute(StyleConstants.NameAttribute);
                System.out.println ("All attrs of " + name + ": " + attributes.getAttributeNames().toString());
                Enumeration e = attributes.getAttributeNames();
                Object obj;
                while (e.hasMoreElements()) {
                    obj = e.nextElement();
                    System.out.println (obj.toString());
                    System.out.println ("attribute of class = " + attributes.containsAttribute("class", "login"));
                }

                if ((name instanceof HTML.Tag)
                        && ((name == HTML.Tag.H1) || (name == HTML.Tag.H2) || (name == HTML.Tag.H3))) {
                    // Build up content text as it may be within multiple elements
                    StringBuffer text = new StringBuffer();
                    int count = element.getElementCount();
                    for (int i = 0; i < count; i++) {
                        Element child = element.getElement(i);
                        AttributeSet childAttributes = child.getAttributes();
                        if (childAttributes.getAttribute(StyleConstants.NameAttribute) == HTML.Tag.CONTENT) {
                            int startOffset = child.getStartOffset();
                            int endOffset = child.getEndOffset();
                            int length = endOffset - startOffset;
                            text.append(htmlDoc.getText(startOffset, length));
                        }
                    }
                    System.out.println(name + ": " + text.toString());
                }
            }
        } catch (IOException e) {
            System.out.println ("Exception?1 " + e.getMessage() );
        } catch (Exception e) {
            System.out.println ("Exception? " + e.getMessage());
        }
    }

The question is: How do I get any element’s attributes and print them out?

Answer 1

This code is needlessly verbose. I would suggest using a better library like Jsoup. Here’s some code to find out all the attributes of all divs on this page.

String url = "http://stackoverflow.com/questions/7311269"
             + "/java-print-any-detail-of-html-element";
Document doc = Jsoup.connect(url).get();
Elements divs = doc.select("div");
int i = 0;
for (Element div : divs) {
    System.out.format("Div #%d:\n", ++i);
    for(Attribute attr : div.attributes()) {
        System.out.format("%s = %s\n", attr.getKey(), attr.getValue());
    }
}

Follow the Jsoup Cookbook for a gentle introduction to the this powerful library.