Struggling mama

My ex just started getting supervised visits with my 2 boys. Less than 4 yrs old. The judge ordered it to be at my house. So im not there but one of my family members is. Im just really struggling with him being in my home. My safety net. Somewhere I made home without him and now hes just acting like hes "owning" the place. He was emotionally and verbally abusive and is a narcissist and I felt like I was doing good until he start seeing the kids in my home. I feel better because its my home. But im struggling just in general because hes already saying things that you don't promise a 4 yr old. And saying things to make it seem like things are my fault. I'm just trying to be positive. And be supportive of the boys seeing their father. But its really hard. Especially when his visitation is on my days off right now. Even tho it's temporary.