I need some serious marriage advice!

Backstory:

A little over 7 years ago, during my first year of college, I met the love of my life, but he had a girlfriend and I didn't act on anything until they were broken up. I've never felt a connection with a person like I did with him. He has my best friend and Oh the passion! I could tell him anything and he told me everything. We were always open and honest with each other. A few months into the relationship he got scared of what we had and decided he wanted to be back with his ex. I was heart broken. I lost my best friend that day. And to top it off the next day I found out I was pregnant. He offered to get back with me so he could support me, I wouldn't let him because in my mind he had already made his decision. I gave him an out. I told him that it was too painful for me to fight for his time when he was with his ex and I didn't want his ex in the babies life. So he let me go. 

When I was 10 weeks pregnant I got food poisoning and my friend, an old boyfriend, came and took care of me. He had always had feelings for me and I was in a really low place feeling like I would be alone forever. And he was such a great guy. A Christian, loyal, great family, and even though I was pregnant he still wanted me. He was there for the birth of my daughter. Took her in as his. He proposed, we got married when my daughter was 1. Should be a great marriage right? It doesn't feel right to me. I don't feel passion. I don't feel that spark, I feel insecure with him. I can't be myself. He doesn't support my career, I'm a Labor and delivery nurse and work nights and he doesn't get it. He's a great dad and provides financially for the family but he doesn't provide emotionally for me. I feel zero passion. I love him but I'm not in love with him. And I've never truly felt 100% like I made the right choice. Before walking down the isle I had this overwhelming desire to turn and run. But I am a people pleaser and would rather suffer myself than others hurt. I've been out of love for a long time. Another thing, kind of the last straw, happened around 8 months ago. We were at an ok place in our marriage. Our second daughter had just turned 2 and I was ready for another baby. He refused, said he would never have another. Completely took me by surprise and after a few months of deliberation he told me we would have another. We tried for a month, and before I tested he said I changed my mind, I don't want another. I ended up pregnant but had a miscarriage and I won't ever forgive him for that. 

I've always felt like something is missing. A few months ago, my daughters dad reached out to me. Instantly I felt all of those feelings again. I feel like I have my best friend back. I want him back in my life. I have forgiven him and hashed out everything about him leaving me and my daughter and I have forgiven him so I don't need an opinion on that. He is my person. I've never felt the way I feel with him than anyone else. He is supportive, he gets me, he is perfect for me. 

I want to leave my husband now more than ever. I am scared. I feel ashamed. I feel like I'm pretending. I do make quite a bit of money bit I'm afraid of losing my house. My dad built this house and I want to raise my children here. I'm also afraid of losing his family. I love his family. And I'm afraid of how my parents will react to me getting back with my ex. But I feel like I have to be happy too. I can't just live a life trying to make others happy. And I don't want to teach my kids to stay in a relationship when they want out. Help I need advice. Has anyone been in this spot?