Question

I am having trouble doing a simple download of an XML file from a URL. I have looked around this site for a while already and followed most of the examples on how to download a file with the proper encoding as far as I can tell, however I must be doing something wrong because I am not getting the desired output. At the moment my code looks like this.

            Catalog cat = (Catalog)obj;

            String datasetURL = cat.getID()+"@datasets";

            URL dataURL = new URL(datasetURL);

            InputStream iStream = dataURL.openStream();

            int count = iStream.available();
            char content[] = new char[count];

            InputStreamReader isReader = new InputStreamReader(iStream,"UTF-8");

            BufferedReader buffRead = new BufferedReader(isReader);

            buffRead.read(content, 0, count);

            String contentAsString = new String(content, 0,count);

            FileWriter fstream = new FileWriter("src/main/resources/datasets.xml");
            BufferedWriter out = new BufferedWriter(fstream);

            out.write(contentAsString);
            out.close();

This seems to work correctly however the xml file is displaying characters like:
�Ksǵ���p� etc in Eclipse and appears as �Ksǵ���Žp� in notepad++. I dont know what to do because I have already added the encoding to the InputStreamReader so I thought that would solve this problem.

Also I am not too familiar with RDF but the xml file has an RDF tag in it. Would that make any difference?

<?xml version='1.0' encoding='UTF-8'?>
<r:RDF xmlns:s="http://www.w3.org/TR/1999/PR-rdf-schema-19990303#"   xmlns:r="http://www.w3.org/1999/02/22-rdf-syntax-ns#" etc.. 

Thanks much.

Answer 1

I found that the problem actually was because the file was zipped! Here is the code I am using now which has successfully downloaded the file.

            Catalog cat = (Catalog)obj; 

            indexName += "."+cat.getInternalID();

            String datasetURL = cat.getID()+"@datasets";

            URL dataURL = new URL(datasetURL);

            URLConnection conn = dataURL.openConnection();

            String encoding = conn.getContentEncoding();

            InputStream is = encoding.equals("gzip")? new GZIPInputStream(conn.getInputStream()) : conn.getInputStream();

            BufferedReader in = new BufferedReader(new InputStreamReader(is));

            String inputLine;

            while ((inputLine = in.readLine()) != null)
                System.out.println(inputLine);

            in.close();

Hope this helps anyone else out there that may be having similar problems.