Question

What is the best practice (interface and implementation) for a command line tool
that processes selected files in a directory tree?

I give an example that comes to my mind, but I am looking for a ‘best practice’:

flipcase foo.txt foo2.txt

could process foo.txt and save the result as foo2.txt.

flipcase -rv *.txt

could process all text files in the current directory.
-r or --recursive will include all subdirectories.
-v will print some infos to stdout while processing.

One problem that I see with this example is, that the *.txt argument is
sometimes expanded by the shell (Unix and Vista), so I can’t apply this pattern
when walking sub directories.
I guess the reason is, that on Unix such tools are comined with a call to find,
but this seems not to be common on Windows. It also makes it hard to print a
summary at the end.

Requirements:

• MUST run on Unix, Windows XP, Windows 7 and Mac
• SHOULD follow common conventions on these platforms.
  (Yes, I know. But I am looking for a reasonable compromise.
  For example it’s Ok to use - instead of / on Windows.)
• SHOULD not rely on a separate find command, like grep does.
• MUST work for single files, file patterns and patterns in directory
  hierarchies.
• SHOULD be build with standard Python libs, e.g. OptionParser and os.walk.
• COULD handle multiple patterns, e.g. *.txt,*.html.

Other questions on design decisions:

• What should this tool return (status code)?
• Which ctrl-keys should this tool handle, and in what way?
• Should stdin be supported instead of a single file? Configurable or
  auto-detect?
• Should output redirection be supported? Configurable or auto-detect?
  How deal with debug output in this case?
• Should the pattern be glob syntax, or a regular expression?
• Is there a common pattern syntax that supports recursion?
  Maybe recursive:*.txt
  In this case the -r option would not be neccesary.
• What is best practice to create backups of modified files?
  Option -b, or rather have backups by default and add --no-backup option
• For single files it should be possible to specify a target file name. How?
• What status info should be printed, and hot configure this?
  Should it be verbose by default and we allow -q for quiet?
  Or always print a little bit and allow -v (or -vv) to boost this or -q to
  shut up completely?

I don’t really expect to get one single right answer, but may be a handful of
thoughts and pointers to good sample projects.

Answer 1

In my experience, the best starting point is to build a tool that follows basic Unix principles — namely, to read from standard input and write to standard output. This allows people to use your tool in a flexible way:

flipcase input.txt > output.txt
othercommand | flipcase > output.txt
flipcase | othercommand > ouput.txt
flipcase input1.txt  input2.txt > output.txt

The next feature might be in-place editing:

# Modify input files directly.
flipcase -i input.txt

# Create backup copies before modifying originals.
flipcase -i --backup-suffix '_BAK' input.txt
flipcase -i --backup-prefix 'BAK_' input.txt

# Regex for power users.
flipcase -i --backup-regex 's/foo/bar/' input.txt

In verbose mode, the tool should not write to standard output, because that would conflict with the core principles above. It should write to standard error or a user-defined log file.

flipcase -v         input.txt > output.txt
flipcase -v log.txt input.txt > output.txt

After that, you add recursive behavior. The direction is less clear-cut here, but I’ll toss out a few ideas. In the typical recursive case, the program’s arguments are probably directories, and the user would need to supply additional options to define various types of filtering behavior (that is, which types of files to process).

flipcase -r -i --backup-suffix '_BAK' --filter-glob '*.txt' dir1 dir2
flipcase -r -i --backup-suffix '_BAK' --filter-glob '*.txt' --filter-glob 'log*.dat' dir
flipcase -r -i --backup-suffix '_BAK' --filter-regex 'log\w+\.(txt|log)$' dir1 dir2

# Don't do in-place editing. Instead create new files within the structure.
flipcase -r --newname-suffix '_NEW'              --filter-glob '*.txt' dir1 dir2
flipcase -r --newname-regex 's/\.txt$/_new.txt/' --filter-glob '*.txt' dir1 dir2

# Create the backups or the new files in a parallel directory
# structure rather than within the original structure.
flipcase -r -i --backup-tree 'backup_dir'   --filter-glob '*.txt' dir1 dir2
flipcase -r -i --new-tree    'newfiles_dir' --filter-glob '*.txt' dir1 dir2