Non-custodial visitation

We went to child support court in July, got payments set up and visitations set up for our kids. When we saw a mediator he was going over visitation and the dad(non-custodial) denied he wanted the kids on Thursday 6-8 because of work and hes only getting them on his weekend like suppose to. So i use to beg him to get kids on Thursday 6-8 he always told me no not getting kids more than i need to and I'm only getting them on my weekends i said OK left it as that. Now all of a sudden dad text me stating he's coming to get kids on Thursday and i cant deny his time and he will bring a officer to take my kids out the home since it's his time. I told him no he cant all of a sudden want to take kids on Thursday now and telling me at last min not knowing if had plans or the kids have school events. Will i get in trouble by not giving him the kids at a last min notice when he never got them like he was suppose to before?

Less