Need advice!!!

My boyfriend acts like he don't want to be around me but when he hangs out with his friends he acts so happy and talks and talks away. But when he's with me he's so quite and don't say nun. I feel that our relationship is going down hill. We have a 3 year old and one on the way. Maybe he just with me for the kids. I'm so hurt and wish things were different. I just want to close him out and not care anymore. I'm trying to tell him.but he don't care. We can't even have a date night to ourself cause he invites his friend all the time!!