I run a mail server on my web server, it has SMTP authentication enabled and I added my server’s IP address to the whitelist, so no password is required to send out emails if they come from within the server. I did this to allow my web applications to use the SMTP service.
My question is can a spammer spoof a packet with my own IP address and bypass the SMTP authentication?
Emails sent from my email server often end up in the recipient’s spam folder and I’m trying to figure out why. I send no more than 200 emails per month.
It isn’t possible to spoof your address unless the attacker is between you and your server (which is very unlikely – he would have to be an ISP or to be able to overwatch and control your internet traffic). This is because after an attacker sends a connection request to your server, the server sends a packet to you that the attacker needs to have to be able to establish a connection: http://en.wikipedia.org/wiki/TCP_handshake#Connection_establishment
Try to look at the headers of a mail that gets into the spam folder and look for something that says that your servers IP is blacklisted or something like that.