Question

as title shows, how do i return a list of urls under (a href) reference and display it in a text file ? The code below return the html form a a website.

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;

public class Main {
    public static void main(String[] args)  {
        try {
            URL my_url = new URL("http://www.placeofjo.blogspot.com/");
            BufferedReader br = new BufferedReader(
               new InputStreamReader(my_url.openStream()));
            String strTemp = "";
            while(null != (strTemp = br.readLine())){
                System.out.println(strTemp);
            }
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }
}

Answer 1

You sound like you want to be using an HTML parsing library like HtmlUnit, rather than getting into the hassle of parsing the HTML yourself. The HtmlUnit code would be as simple as:

final WebClient webClient = new WebClient();
webClient.setJavaScriptEnabled(false);
final HtmlPage page = webClient.getPage("http://www.placeofjo.blogspot.com/");

//  Then iterate through
for (DomElement element : page.getElementsByTagName("a")){
    String link = ((HtmlAnchor)element).getHrefAttribute();
    System.out.println(link);
}

Gives output of:

http://www.twitter.com/jozefinfin/
http://www.facebook.com/jozefinfin/
http://placeofjo.blogspot.com/2008_08_01_archive.html
... etc etc
http://placeofjo.blogspot.com/2011_02_01_archive.html
http://endlessdance.blogspot.com
http://blogskins.com/me/aaaaaa
http://weheartit.com