Question

I have created a JSP code where we can upload a .csv file. The JSP Code is supported by a java code that reads the .csv file and compares the urls in the file with the DB and adds it into to the DB if the urls are not already present.

The above scenario works absolutely fine when its executed in a windows system.

I uploaded the succesfully executed web application folder to a unix system. When I executed the program in the UNIX system, the tool is not comparing the URLs with the DB and adds it.

I suspect there should be some problem in reading the .csv file in a UNIX sytem.

Am using fedora(linux) OS. Kindly let me know whether there is any differences in reading .csv file between a windows system and a unix system.

The .csv file I am using has the following contents,

http://www.topix.com,sdfasdf
http://rss.news.yahoo.com/rss/topstories,Apple
http://www.apple.com/354,sdfasdf
http://www.topix.com/rss/city/emporia-ks,sdfasdf
http://www.topix.com/rss/,sdfasdf
http://ax.itunes.apple.com/WebObjects/MZStoreServices.woa/ws/RSS/topsongs/limit=10/xml,sdfasdf
http://www.topix.com/rss/city/emp,sdfasdf
http://www.topix.com/rss/city/sandy-ut,dfgsdfg
http://www.apple.com,Yahoo

UPDATE FOR JEFF

try {
                    List items = uploadHandler.parseRequest(request);
                    Iterator itr = items.iterator();
                while(itr.hasNext()) {
                    FileItem item = (FileItem) itr.next();
                    if(item.isFormField()) {
                        out.println("File Name = "+item.getFieldName()+", Value = "+item.getString());
                    } else {
                        File file = new File(destinationDir,item.getName());
                        item.write(file);
                    //String temp=item.getName();
                        String fileToBeRead = "C:/Program Files/Apache Software Foundation/Tomcat 5.5/webapps/Readcsv/files/"+item.getName();
                        String urlcnt="";
                        String srccnt="";
                        String contentType="";
                        Connection con=null;
                        Statement stmt=null;
                        final String rssvar="Rss";
                        final String other="Other";

                        int i=0;
                        int j=0;

                        try {
                            BufferedReader br = new BufferedReader(new FileReader(fileToBeRead));
                            String strLine = "";
                            StringTokenizer st = null;
                            while( (strLine = br.readLine()) != null)
                            {
                                st = new StringTokenizer(strLine, ",");
                                while(st.hasMoreTokens()){
                                    urlcnt=st.nextToken();
                                    srccnt=st.nextToken();
                                }
                                if(con==null){
                                    SQLConnection.setURL("jdbc:sqlserver://192.168.2.53\\SQL2005;user=sa;password=365media;DatabaseName=LN_ADWEEK");
                                        con=SQLConnection.getNewConnection();
                                        stmt=con.createStatement();
                                }
                                try{
                                    ResultSet rs;
                                    boolean hasRows=false;
                                    rs=stmt.executeQuery("select url from urls_linkins where url='"+urlcnt+"'");
                                while(rs.next()){
                                    hasRows=true;
                                    i++;
                                    }
                                if(!hasRows){
                                    j++;
                                    URL url = new URL(urlcnt);
                                    URLConnection url1=url.openConnection();
                                    contentType=url1.getContentType();
                                    PreparedStatement insertUrlStatement = con.prepareStatement("INSERT INTO urls_linkins(url, source_name, is_active, is_periodic, Link_Type, New_Entry) VALUES(?, ?, ?, ?, ?, ?)");
                                if(contentType.contains("rss") || contentType.contains("xml"))
                                {
                                    insertUrlStatement.setString(1, urlcnt);
                                    insertUrlStatement.setString(2, srccnt);
                                    insertUrlStatement.setInt(3, 1);
                                    insertUrlStatement.setInt(4, 0);
                                    insertUrlStatement.setString(5, rssvar);
                                    insertUrlStatement.setInt(6, 1);
                                    insertUrlStatement.executeUpdate();
                                    insertUrlStatement.close();
                                }
                                else{
                                    insertUrlStatement.setString(1, urlcnt);
                                    insertUrlStatement.setString(2, srccnt);
                                    insertUrlStatement.setInt(3, 1);
                                    insertUrlStatement.setInt(4, 0);
                                    insertUrlStatement.setString(5, other);
                                    insertUrlStatement.setInt(6, 1);
                                    insertUrlStatement.executeUpdate();
                                    insertUrlStatement.close();
                                    }
                                }
                                }
                                catch(Exception e){
                                    e.printStackTrace();
                                }
                            }

                            }catch(Exception e){
                                e.printStackTrace();
                            }finally{
                                out.println("<h2>"+j+" url has been added and "+i+" url already exists in the DB</h2>");
                                out.println("<a href=Addurl.jsp>Check URL</a>");
                                out.println("<a href=Addurl1.jsp>Add Single URL</a>");
                                out.println("<a href=uploadcsv.jsp>Add Multiple URL</a>");
                            }

                            }
                                out.close();
                            }
                            }catch(FileUploadException ex) {
                                log("Error encountered while parsing the request",ex);
                            } catch(Exception ex) {
                                log("Error encountered while uploading file",ex);
                        }

This is my reading code of the .csv file.

Answer 1

Yes there will be differences in reading the .csv file when you transfer from a windows machine to a unix machine even when it’s a text file. There are hidden space characters which may be represented differently on the unix machine.

I suspect that the reason it is not comparing the URLs is because the space characters might be different ASCII values so it thinks they are different and adds the URL into the DB.

One suggestion would be to use the dos2unix command.

http://kb.iu.edu/data/acux.html

Hope it helps.