Should I leave or should I stay?

I feel so confused... And kind of hurt I guess. I've been with my boyfriend for 6 years, we have a little boy who will be 2 on October 21st. We've been trying to conceive baby #2 for about a 3 months now... But lately he's been so distant... I'm still breastfeeding my son so I'm the only one who can put him to sleep and when I finally get him to sleep and go to bed he acts like he's so tired 😭 there's even times where he acts like he's so tired from work but the minute his friends call to hangout he jumps up...  sometimes I feel like I'm just being over emotional and then there's days where I feel as if I'm completely right for feeling hurt!!! And when I try to talk to him about it it never gets anywhere, he just tells me he needs time alone and to stop over reacting, but I really haven't been, I don't argue with him, or get mad when he goes with his friends.. I'm just tired of going in circles with this man.. I feel so confused and lost.. What do you lady's think?