If I call System.IO.File.Delete() and pass it a directory name I get UnauthorizedAccessException although the problem has nothing to deal with any kind of permissions and I guess InvalidOperationException would be much more suitable here.
Why is this specific exception being thrown? Is it a design fault or is there any reason for that?
No idea, but it smells like a design fault because the Directory.Delete has a different behavior that File.Delete. Directory.Delete will throw a DirectoryNotFoundException if the path actually refers to a file instead of directory. One would expect that the same kind of exception would be thrown in these two cases.