Definitely look into heavy metal toxicity. Research Andy Cutler, DMSA, ALA, DMPS, etc.
"I do eat whole grains, dairy, beans,"
Too many carbs. And eating meat prevents die-off. Try eating for 1 week all green leafy veggies (JUST the leaves, no stems or roots) and taking a supplement called candex which digests the cellular wall of candida. This worked for me when nothing else did.