Question

I’m new to Java, so bear with me if I say anything stupid! I’m having a few problems, which I think are Unicode-related.

I’m using Scanner to read in tokenised commands from a text file, saved with UTF-8 encoding. Basically I want to first check that the command isn’t equal to either “command1” or “command2” (I do something else in these cases), then otherwise read in a character. If the token isn’t a single character, I’m going to output an error.

Here’s my code:

public static void main(String[] args) throws FileNotFoundException {
    Scanner scanner = new Scanner(new File(args[0]));
    while (scanner.hasNext()) {
        String command = scanner.next();
        if (command.equals("command1")) {
            System.out.println("command: command1");
            // do something
        } else if (command.equals("command2")) {
            System.out.println("command: command2");
            // do something
        } else {
            if (command.length() == 1) {
                char c = command.charAt(0);
                System.out.println("character: " + c);
                // do something with c
            } else {
                System.err.println("error (string was " + command
                        + " with length " + command.length() + ")");
            }
        }
    }
}

And the contents of the text file whose filename I’m passing in args[0] for testing:

command1
x
y
command2
z
└
command1
╒
═

Expected output is:

command: command1
character: x
character: y
command: command2
character: z
character: └
command: command1
character: ╒
character:  ═

Actual output is:

command: command1
character: x
character: y
command: command2
character: z
error (string was └ with length 3)
command: command1
error (string was ╒ with length 3)
error (string was ═ with length 3)

As you can see, the non-standard characters are being seen as a 3-character string by Java. Strangely, if I copy/paste the one of the characters from the terminal output into a System.out.println("└".length()) statement, it correctly prints 1.

Any ideas on where I’m going wrong?
Thanks

Answer 1

When you open files in Java, the encoding (if you don’t specify one) is taken from the file.encoding system property. This is almost never set to something that you want (if you’re like me, you always want UTF-8).

To fix, explicitly specify your character set when you create your Scanner:

Scanner scanner = new Scanner(new File(args[0]), "UTF-8");