Losing friends because of age gap love.

My husband is 41 and I am 27. We've been together for 7 years, married for 6 months now. I love him more than words can describe. We both have social anxiety disorder (we met on an anxiety website) and it's difficult for us in social settings generally.

Well he doesn't care to make friends but I do. I've met a couple of women closer to my age, and it never seems to work out. Once they meet my husband they began to act odd around me or never contact me again. I guess I can understand their point of view....hanging out with someone older might not be their thing....but why cut me off! :(

anyway....I've started to get close with this girl at work...like we're becoming really good friends and she wants to have a double date...she 22 and so is her boyfriend. I guess I'm scared of being rejected again and I kinda don't want to do the double date, but I am not ashamed of my husband. I guess if it doesn't work out it just doesn't. Ugh. Anyone else gone through this?

Eta: no my husband is NOT creepy or make others feel uncomfortable. If anything I am the one who can make others uncomfy. My husband is very kind, and can hold conversations better than me. When we've been on double dates I am always near him so I haven't witnessed anything to come to that kind of conclusion!