Problem maintaining an erection when I use condoms

I'm 23 years old and I've only engaged in intercourse with a few women, so I'm not sure how commonplace this is. The first time I was with a girl I foolishly did not use protection yet I managed to perform pretty well. Months ago I was with another girl, however, and used a condom then. It worked for a short while but then I began to lose my erection. Just a few days ago, I was with my new girlfriend for the first time and the same thing happened. I'm pretty worried as I have no idea what's wrong with me, if this is normal, or if there's anything I can do about it.

I'm in good shape as I exercise a lot, so I don't think it's because I'm unhealthy. I'm wondering if it's because of my self-gratification habits (I can spank the monkey anywhere from 1-3 times a day on average, sometimes skipping a day or so in between). Oral sex doesn't seem to do too much for me either, as I've never had an orgasm that way. Is this normal? Tweet