Advice please

So my boyfriend and I have had a pretty good relationship so far. He has 3 kids but only 1 was living with him and I have 2 kids but again only 1 lives with me. Everything was going fine up until his ex wife calls him and tells him that her boyfriend beat her up and he needs to drive 3 and half hours to pick up his kids. So we go to pick them up and bring them home but ever since then he wants nothing to do with me my son and his daughter that had been living with us. When the ex had the kids she wouldn't let him talk to them but she calls his phone at least 3 times daily and most of the time its to talk about something other than the kids. Every since he had been talking to her everyday he has been fighting with me all day everyday! Can somebody please tell me if I'm over reacting about the ex calling all the time and what could possibly be going thru his mind to always be fighting with me but doing everything his ex says!?