Question

i have csv files, java app and database, i read csv file from my java application and
after some operation i insert files, one by one, into the database, but characters like ŠĐŽĆČ aren’t inserted correctly. I set database to utf-8 default collation. I realize, through testing, that when i set files with ANSI encoding everything works just fine(ŠĐŽĆČ are inserted correctly into database) but when encoding is UTF-8 characters aren’t inserted correctly. Problem is that the files , i have to insert into DB, must be encoded in UTF-8.
Can you help me with this problem?

Answer 1

Difficult to be specific without more details, but there are a couple of things which spring to mind:

1) If I understand you correctly, you are saying that your CSV are in UTF8? Have you verified that assumption is correct? And on a related note, given that you think it is UTF-8, how are you loading that file into Java?

Note, Java’s file readers use the system’s default character encoding. It’s not necessarily UTF-8, and you can check if you print System.getProperty("file.encoding");.

I tend not to rely on this and load according to a known encoding as follows:

FileInputStream fis = new FileInputStream("myfile.txt"); 
BufferedReader reader = new BufferedReader(new InputStreamReader(fis, "UTF-8"));
...

2) Database collation is different to specifying a character set. (Note, it would be handy if you also mentioned your database client). Collation relates to sorting/ordering of text. You generally need to specify the character set. How this is done depends a lot on your database.