Am I crazy?

So long story, but I want opinions.

I just recently (10/2) got divorced (we were separated for a while, and I've been seeing someone for almost a year now) and we have standard visitation that must be supervised as my ex has a history of violence (he pleaded guilty to a domestic violence charge in July). Since the divorce my son has been over for his weekends 3 times and his father has taken him to the emergency room twice. The 1st time he called me and I went with them - there was nothing wrong with my child. In the final order of our divorce it states that we half all medical bills. I have to work 2 jobs to support my child. His father doesn't pay child support, doesn't help with clothes, food, nothing. I've come to accept this as what it is, but I know that deep down he is taking him to the dr because he will not pay his half of the bills and he is trying to stick me with all the cost. This weekend was his weekend. I received a call at 9:30am on Saturday from the local hospital following up with the visit my son had the night before. His father did not call me. I tried to phone his father and he didn't answer, so my boyfriend drove me to his house so I could check on my son. When my ex answered the door I asked why he took my child to the hospital without telling me and he replied it was because he's "covered in bite marks and bruises" (he is not, but he's called CPS on me prior to this because my son fell down on a vacation and fractured his collarbone, so I know exactly where he was going with this.) When I asked again why he didn't call me, he became irate and when my boyfriend stepped inbetween my ex and myself my ex attacked him. My son was standing in the doorway screaming (mind you, this is the 4th time in less than a year that he's become violent in front of him and this makes the 2nd time he's broken his probation). So I took my child, placed him in my car and we left. My boyfriend is pressing charges against my ex. 

I emailed my lawyer about this matter because at this point, I'm concerned for my sons well-being and safety when he's over there and she just basically told me I went over there looking for a fight and I should have waited until I got him back to check and see why he went to the hospital. That response is making me feel like I'm crazy. I feel like any mother would've been concerned. But maybe I'm wrong? Would any of you went to check on your child? Would you have also taken him away even though it wasn't your weekend? 

Edit; pictures added if the bruises he took him to the ER for.