You have a right to your choice, which I feel is very wise and mature. Casual sex is vulgar and embarrassing, sex is meant to be a very spiritual act. And you don't have to worry about unwanted pregnancy or catching diseases, not to mention the emotional toll it can have.
I've noticed that polygamists are hardly ever in favor of polyandry, which is a woman with multiple husbands, why the double standard?
Best wishes to you, CF, on whatever path you choose to take.