Normally, square matrices are used to transform various points. You have a 4D vector which transforms into a 1×4 or 4×1 matrix, which is multiplied by a 4×4 matrix on whichever side works to create a 4×1 or 1×4 matrix. This is your transformed point.
But what are non-square matrices used for? If you multiply a 3×4 matrix by a 4×1 vector matrix, you end up with a 3×1 matrix; that’s one less dimension than what you started with. Not to mention that a 3×4 matrix doesn’t have the nice properties of a 4×4 matrix such as the ability to multiply by other 4×4 matrices and still retain the same dimensions.
GLSL supports both square matrices and non-square matrices. Why?
You can use 4×3 matrices for, for example, skeleton bone transformations in skinning. These transformations are affine and don’t include projection, so you don’t need the last column.
The result of the multiplication of 1×4 vector and 4×3 matrix is a 1×3 vector, and in this particular case you don’t need to multiply it with a matrix.
Passing 4×3 matrices to GPU instead of 4×4 will save you a quarter of bandwidth.