You will get this reaction with anything that you do that is different or especially following God. I've had rumours spread about me because I had so many children. I've had a lot of people make some weird assumptions about me, or tell other people false things that they truly believed were real.
Nearly everybody disagrees with me and my husband about something. Sometimes they make those things huge, rather than understanding we're all different and God asks us all to live differently. There's a lot of judgement.
Here's what you need to know about judgement, it has nothing to do with you. It comes from their own fears or jealousy or guilt.
For example, I had a woman come around with her children for a play date. I didn't know her very well but I invited her around. We had just finished school for the day, and I was explaining how we had been unschooling that day, and she became very quiet and asked if I judged those that didn't homeschool. I was surprised but glad she asked the question. Of course I don't judge those that don't homeschool. Everyone should do what's best for them and/or their children, I truly believe that. I'd never said anything that had made her think I felt that way, I'd just talked in passing about our day etc, because it's a big part of what I do each day, but she'd been sitting there thinking it. Where she was coming from, was that she had wanted to homeschool her children. She had thought about it, and her and her husband had decided against it. She was feeling guilty about that decision.
For some bizarre reason, people seem to think that because I live a certain way, and feel God has told me to live a certain way, that I think that everyone has to do it this way or I'm sitting home stewing and judging them for their 'poor' decisions. Nothing could be further from the truth.
Now, this comes into play in a big way in PM. because the moment it's known about you, or you bring it up, they start questioning. If it's not a sin, then their husband could just take another wife, couldn't he? And that would be the end of the world. So it's fear. And because you're happy and showing it can be done, it makes that fear worse, so they judge you, and one of the ways they do that is by spreading rumours, and by saying that you're actually judging them because every time you talk you are trying to prove it's right (because you're defending yourself...).
Make sense?
So, how do you deal with this? First of all, you acknowledge that it has nothing to do with you or your actions. Then you pray for the Holy Spirit to work on their hearts and break through the fear. Know that you're not alone, others have been through this too, and are going through it right now. Stay strong in the truth, support each other as a family, and keep close to God, always.