Hello group I apologize this is long.

I have a problem. My ex boyfriend- guy I lost my virginity to- is still in my family. We dated for about a year and a half back in 2010-11. We broke up because I left to join the military. Well during the time together my parents helped him with a lot of things, he got into college because of them, he got a car, insurance, a scholarship. During these last 6 years he's remained a part of my family, in Christmas cards, at family gatherings; my parents help him with money issues, car trouble, school problems everything. He is an ex to me and ive told my parents so many times he's not my family hes my ex. Now I'm married and expecting and my husband hasn't had to really deal with it because my ex has never been around. So I go to my first family camping trip in 5 years this year, my mom doesn't tell me my ex will be there until we got on the interstate and couldn't turn around. Then today I get home from work (I'm transiting to Florida and living with my parents temporarily) and there he is. In the house no notice about coming or anything. That's twice now that he's caused a fight with my husband and I because my husband finds it extremely disrespectful that he still comes around. I've put up with my ex being in my family for 6 years because It makes my parents happy and I just deal with it. But now I'm not sure what to do. Do I tell my parents it has to stop and if he's around then I will not be because my husband is uncomfortable with it? Or do I tell my husband that its not gonna change and he has to get over it and accept him in the family?

Thanks for the help!