Need advice

So my fiancé has 3 kids with another woman. Me and him have a 5 month old. Him and his ex meet at a Walmart in between where we live and where we live. Back story here is he takes the kids when she has plans even if we have plans he knocks them off, he will pick them up at her work 40 minutes from where we are without telling me, he goes where she tells him to and still does stuff for her not regarding the kids. He doesn’t talk to our daughter unless I ask him to, he doesn’t help me out with anything or our daughter with anything. Am I in the wrong for feeling upset and in the corner about this? I need advice please