Mother is upset about my decision

I hurt my moms feelings by telling her my baby can’t go over there .. so here’s a bit of a back story, I’ve never liked my mothers husband & he’s never liked me .. I got kicked out when I was 15 because my mom started dating him & he didn’t like stuff I did & he got in my moms ear about it. Well I’m now 23, pregnant with my first child & her first grandbaby. about a year ago my sister called me bc my mom & step dad got into a fight & I went & picked my sister up & my step dad came running out calling me a fat ass, a pig & so on.. so I went to go confront him, he then went to go run out & hit me until my boyfriend ran over to me .. & showed my mothers husband a gun & he went to go take it from my boyfriend so my boyfriend pointed it at him & then he hid behind my mom & started calling him Mexican racial slurs .. & telling him to go back to Mexico. (He’s half Mexican) & calling him a gang banger.(he’s in no way shap or form part of a “gang”) my mother & her husband then called the cops on my boyfriend. cops came, my boyfriends gun is legal & could clearly see my sister crying & my moms husband still yelling racial slurs & saying “show the cops your gun now dumbass, bet it isn’t legal.” The cops told us to go home & we are fine, gave us his gun back. my moms husband then started texting my boyfriend telling him that he’s gunna beat his ass, calling him racial slurs. Which then lead my boyfriend to blocking him. Now fast forward to me being pregnant & my mom still being married to him.. she was talking to me the other day & the topic came up on the baby going there & I told her no, the baby will not be going to your house.. I told her that her husband was racist & has been verbally & physically abusive to you to where he’s head budded you .. I’m not having my baby around that.. my baby still is part Mexican, & your husband doesn’t like me nor my boyfriend. so no.. that will not be happening but you can come over & see the baby & go out with the baby & I & take the baby out but the baby will not be allowed at your house. To where she then got really upset & left. Am I wrong for feeling the way I feel? like I don’t want my baby around that.. & I don’t trust him..