Jealous?

So last year my fiancé cheated on me with his ex and ended up getting her pregnate but I had no idea then I found out and I tried leaving him but a week after I found out I was pregnant too. By this time his ex was already 9 months so now I can't leave and he tell me I'm the one he want to be with we live together. And then he has his baby with his ex and she said she doesn't want me anywhere near her daughter so he has to stay at his moms house Thursday through Saturday afternoon to watch her. And I've never been ok with this he just leaves and then I don't see him until Saturday... Well I've told him lots of times I'm not ok with this and it seems like he just ignores me I get it his daughter is more important and he can't go to court because he had no papers so if he breaks their arraignment because of me she will take her daughter away from him. So today he says he forgot his charger and I go and leave it at his moms and I see her inside with his mom chatting away and he's there. And he swears she was just there dropping off clothes but really like everyone is ok with this everyone but me I'm 37 weeks pregnant and I can't be at their house when she's three and I'm so fed up with seeing this I feel like such an exclusion I'm thinking of leaving him when I have this baby if things don't change.