Why do you worry so much what people think? Worry about what Hashem thinks.
Moses married a Cushite woman from the tribe of Midian!
Yeah, but I know.. I know. I really don't care what other people think to a large extent, otherwise I would have never converted. It only when I have to consider other people in my life that I have to bother with protecting their feelings, defending their good standing, and stuff like that. That's just how I am. If I say that I've got your back, then I've got your back. I do care about the Rabbi and would not want to behave in a way that would invalidate the good work he does or my status as a Jew. I'm well aware that we're free to do a lot in our lives as Jews, just so as we conform to the Torah.
But with his whole "I have to marry a Jew" thing gets so complicated. It always undoubtingly goes back to the "Who's a Jew?" thing, along with "Are you with us, or are you with them?" Compounded by the fact that at this age, I hardly ever meet any Jewish women that I connect with. They're either overbearingly hostile to Judaism, not interested in having children, too old to have children, or they sway "to the other side of the fence". Plus all the other usual stuff that happens when people from different backgrounds get together.
But what can I say? It's just frustrating.