I feel like I’m being rude but idk what to do

(Sorry if this is long) Growing up my bf never had the best relationship with his mom based on what he’s told me and spent most of the time living with his dad. We have been dating for 2 years and over the past year he has gone to visit her once a week or so and has invited me to come along. The problem is that I feel really uncomfortable when I go to her house. Not with her or the people there, but with my surroundings. She’s a bit of a hoarder, so there is stuff everywhere to the point where there is barely room to sit. I get very claustrophobic so this gives me anxiety. She also had a ton of pets running around that I’m allergic too, so I always feel sick when we go over. I keep making excuses to not go over there but I hate that I have to. Do I tell my bf what’s bothering me? I don’t want to come across as rude or snobby for not wanting to go over to her house because it’s his mom and it’s nice that she even invites me over :(