Don't know what to do

Ok. So my step son has been living in Florida with his mom. We just got him yesterday for the rest of the summer which I'm super excited about but I'm struggling with two things..so we have been ttc for about five months now and having him around is just a constant reminder of that fact that it took his mom one try and it's taken me 5....but I can get over that. What frustrates me the most is that my husband is like separating us. For example, last night I said something about how he's such a softy when it comes to his son kinda half teasing him. His response was "well when you have your own child that comes from you you'll understand. It's different"...like thanks.then this morning I could have walked out the door and he would have never noticed. He almost didn't but I said I was leaving before I walked out the door for work. I don't want us to be separate. I don't want it to be my husband and his son and then me. I want to be one unit especially for when we do finally have more kids. I'm not trying to be selfish but being a parent and a spouse is a balance and if I have to go my whole summer being the third wheel I don't know what I'll do...