Hanging out with his babymama boyfriend.

so to make a long story short, my baby daddy has a 5 year old daughter from a previous relationship, and we have a 1 year old son together and expecting a daughter together on the 24th of September. I'm aggervated because every time he goes and picks up his daughter he has to spend hours by her house having buddy buddy conversation with her man. He tells me that they are not really cool like that but he is always spending hours over there in MY CAR. (RIGHT NOW WE ONLY HAVE ONE CAR) and it bothers me because he'll tell me that his going pick up his daughter but don't come back untill hours later! like it bothers me that he doesn't at least call and tell me that like "hey I'm going to be over here for a while kicking it" no, I just have to sit back and wait for him to come home. And he gets mad when I call and ask him what's taking so long like he feels like he is a grown man and he doesn't have to let me know that, but I feel like out of respect for our relationship that he should at least tell me that. what do you guys think?? and his other baby mama feels so comfortable coming over to vent to him about her realationship I think that is so disrespectful, she says she has no female friends and that she needed to vent to him. I'm just so pissed because while he is over there playing doctor Phil he doesn't know that his own realationship is suffering because of that. what should I do?