Why do you have to take your ex back when he's been insensitive and does not love you for what you are? He told you in your face that he's only using you for sex. So don't let someone trample your self-esteem like you have no feelings at all. Ignore the ex and get on with your life. He's not the only man around! End all communication with him by closing that email account and changing phone numbers. It would be better if you find a new place to live in so he will not be able to visit you.