second thoughts?

I'm 34, I have two boys who are 11 and 14. I didn't plan on having anymore children, I had my tubes clamped when I was 25. Now I'm re-married, I didn't plan on falling in love with someone who has no children of their own, but I did! so now I had a reversal surgery and I'm good to go ttc. At first I was ecstatic to give my husband a child of his own (even though he sees my two as his own) Sometimes I have second thoughts. I'm an advid gym lover, I spend 2 hours a day lifting weights, my children are at the point now I have so much freedom, and now that we are ttc I no longer drink alcohol or smoke a bowl here and there (but he still does). Thinking about giving this all up is giving me second thoughts lol. I feel like I'm being selfish, and I don't want to be. it's just I feel my husband is not going to give up as much as I am. Does anyone feel the same way? I am really hating myself for feeling this way. What should I do?