Question

In the process of creating a small CLI notebook application, I decided to go with the cmd Python library (see also cmd on PyMOTW).

My shell is UTF-8.

→ echo $LANG
fr_FR.utf-8
→ echo $LC_ALL
fr_FR.utf-8

And it is working quite well.

→ echo "東京"
東京

Starting the code of my little app and trying to use utf-8:

→ python nb.py 
log> foobar
2013-01-15 foobar
log> æ±äº¬
2013-01-15 æ±äº¬

Edited The expected input/output is. When I type utf-8 characters, be accent or Japanese characters in that case, I get garbage.

log> 東京
2013-01-15 東京

So when starting the program the command line changes the type of the input.

#!/usr/bin/env python2.7
# encoding: utf-8
import datetime
import os.path
import logging
import cmd

ROOT = "~/test/"
NOTENAME = "notes.md"

def todaynotepath(rootpath, notename):
    isodate = datetime.date.today().isoformat()
    isodate.replace("-", "/")
    return rootpath + isodate.replace("-", "/") + "/%s" % (notename)

def addcontent(content):
    logging.info(content)

class NoteBook(cmd.Cmd):
    """Simple cli notebook."""
    prompt = "log> "

    def precmd(self, line):
        # What is the date path NOW
        notepath = todaynotepath(ROOT, NOTENAME)
        # if the directory of the note doesn't exist, create it.
        notedir = os.path.dirname(notepath)
        if not os.path.exists(notedir):
            os.makedirs(notedir)
        # if the file for notes today doesn't exist, create it.
        logging.basicConfig(filename=notepath, level=logging.INFO, format='%(asctime)s - %(message)s')
        return cmd.Cmd.precmd(self, line)

    def default(self, line):
        if line:
            print datetime.date.today().isoformat(), line
            addcontent(line)

    def do_EOF(self, line):
        return True

    def postloop(self):
        print

if __name__ == "__main__":
    NoteBook().cmdloop()

So I guess there might be things to override in the original Class of cmd. I checked the module but without luck yet.

Edit 2: Added LESSCHARSET as recommended by @dda

LANG=fr_FR.utf-8
LANGUAGE=fr_FR.utf-8
LC_ALL=fr_FR.utf-8
LC_CTYPE=fr_FR.UTF-8
LESSCHARSET=utf-8

Answer 1

Your code works perfectly for me, Karl. See this:

dda$ ./nb.py 
log> tagada
2013-01-15 tagada
log> 香港
2013-01-15 香港
log> 

And the notes.md file contains the proper entries. So I don’t think it’s cmd that’s at fault here, but probably something in your terminal settings. Try adding

export LESSCHARSET=utf-8

in your .profile.