Question

I want my Java application to download CSV files from http://www.nasdaqomxbaltic.com/ for example or any other site that offers data files.

How can I tell my java application to change from and to values before downloading a csv file?

The following is form for choosing date range : http://www.nasdaqomxbaltic.com/market/?pg=charts&idx_main%5B%5D=OMXBBGI. There are default from and to values assigned by website at the moment and I get CSV file including data between default from and to date.

Could you suggest any java library that can change from and to values in order to download csv file including only particular date range?

Answer 1

First, you can direct search by date with the following URL :

http://www.nasdaqomxbaltic.com/market/?pg=charts&lang=en&idx_main%5B%5D=OMXBBGI&add_index=OMXBBPI&add_equity=LT0000128266&period=6months&start_d=19&start_m=7&start_y=2011&end_d=19&end_m=1&end_y=2012

where you can give the wanted date values to these parameters:

ex :start_d=19&start_m=7&start_y=2011&end_d=19&end_m=1&end_y=2012

Then the idea is to get the rendered web page. Here, you can use the Jsoup library like this :

Document doc = Jsoup.connect("http://www.nasdaqomxbaltic.com/market/?pg=charts&lang=en&idx_main%5B%5D=OMXBBGI&add_index=OMXBBPI&add_equity=LT0000128266&period=6months&start_d=19&start_m=7&start_y=2011&end_d=19&end_m=1&end_y=2012").get();
Elements csvLinkFiles= doc.select("div.download > a");
int filesCounter = 0;
for (Element link : csvLinkFiles) {
    String href = link.attr("href");
    URL csvFileLink= new URL(href);
    ReadableByteChannel rbc = Channels.newChannel(csvFileLink.openStream());
    FileOutputStream fos = new FileOutputStream("csv-file-"+ (++filesCounter) + ".csv");
    fos.getChannel().transferFrom(rbc, 0, 1 << 24);
}

I’m not sure about the code, but the idea is here.

EDIT :

After performing a little request, it seems that you have to perform an HTTP Get (because the file is not static and is returned on request).

Here is a little piece of code that should replace all the code below the URL csvFileLink= new URL(href) line :

GetMethod get = new GetMethod(href);
InputStream in = get.getResponseBodyAsStream();
File file=new File("csv-file-"+ (++filesCounter) + ".csv");
OutputStream out = new FileOutputStream(file);
int read = 0;
byte[] bytes = new byte[1024];
while ((read = inputStream.read(bytes)) != -1) {
    out.write(bytes, 0,  read);
}
in.close();
out.flush();
out.close();