Husband vs. family

Ok, so my family has a lot of problems. My brother has been in and out of children's mental health residential programs since he was 10. He's now 16. My mom has many mental health issues and moved out of the house from her husband and son a year ago. She also has been in the psych ward multiple times. For the past 10 years my mom was emotionally and verbally abusive to me and my siblings. I refuse to see my step father. My mom asked if she could meet me and my son with my brother this week. The zoo or something. I want to have a relationship with them, but I know my husband doesn't want our son around them. (16 months old). I can't believe I'm asking this, but do I go and tell my husband after. Or do I "ask" him if I can go?