Am I too sensitive or is he mean?

So I had to go get a drug test done for school and I said “we’re picking abby and nora up” (my cousin and her baby) my bf said “why?!” And I told him that they were coming over to visit and to give my poor cousin a break. She’s a really struggling single mom. So he says “okay but you will watch him the whole time don’t bring him to me once”(talking about our son). He said it very rudely. My idea was to watch Nora so Abby could shower eat and just chill. But I felt like he wanted to punish me for having my cousin over. I don’t mind keeping my son with me he just acted like he wouldn’t watch him because I had made him mad.