Is there any reason (besides having physically different looking icons) to include four different sizes of my launcher icon? I would think if I just included one icon in my app (in xhdpi directory), the ldpi devices would just use the xhdpi icon and scale it appropriately.

Correct. And you have absolutely no control over the quality of the result.
So, if you try this for an image (launcher icon or any other) and if the results are acceptable on all major screen densities, go for it. However, there may be some images where the degraded results from the downsampling means that you will want to supply your own images at other densities, where you have more control over the results.