I grew up going to church, and I think I understand your feeling here. You are feeling that it's a source of support, and you remember it fondly from the days when you attended and were accepted.
I would just like to point out to you, that what they offer is NOT fellowship, but is rather close to temptation. The fellowship they offer comes at the price of the truth, and caving into to man's pseudo morals in the marriage dept. The support they offer is conditional, and so really isn't support at all, and if you have to worry about it being recalled and taken back because of man's preferences.... why would you want it?
We don't currently attend anywhere, because we KNOW they wouldn't like us for long. Poly is only one of the areas where our views and not mainstream, and as soon as we were honest about the beliefs we do hold, it would be over.
I know though, that for myself, I would much rather have the confidence that comes with knowing what I believe, and why I believe it, and the surety that the truth is the way to freedom.
I wouldn't trade what I have, for the fearful, "have to worry about offending someone" sort of religion that most people seem to be living in. And don't get me wrong, I'm not out there trying to offend, but I have realized that people who are offended at the truth, are not the sort that have the staying power for conflicts, or the strength of character that makes them loyal and trustworthy. In short you can't count on them anyway, and it's better to know that before you get into a bad situation.
We still have contact with friends and neighbors that attend church, the only thing we are "missing" is the worry of being kicked out of "their" church.
Not a bad place to be in my opinion.
Really, what you are giving your children, by setting an example of living by the word, and standing on your own convictions, is worth so much more then what they would have if you just went with the flow.