I looked up the msdn documentation and it says that InvalidUserName is thrown when it does not find the username in the database, which is fine because the user I am creating should not exist in the database.
If I use test@example.com, it works, but if I try it with test.@example.com, the status from Membership.CreateUser is InvalidUserName.
Email addresses come in the form username@domain where domain is at least two parts separated by a period/dot.
The RFC Specification says that a . in the username part of the address (the part before @) is valid unless the . is the first or last character (of the username), so the behaviour that you are noticing is correct.
https://www.rfc-editor.org/rfc/rfc5322
Good find @David Stratton: From the wikipedia link in his comment…