I had the same problem at one time and my prolactin level was through the roof. Apparently I had thrown it out of balance due to my consistent use of 5-Htp which can apparently increase prolactin levels. I stopped it, started using Tyrosine Daily as well as Vitex (Chaste tree berry) and my levels returned.
I've learned since that Vitex is probably best avoided by men, but in this situation it could be of some help temporarily. Best of all after the month passed I was back in normal range and I didn't have to go for an MRI or anything else that could potentially cause problems.