Argument with husband over the in laws

My husband and I have been married for 15 years. His parents have only came to our house maybe a total of 10 times in those 15 years. We have a 8 year old daughter that barely knows them because they never come to see her, we've took her over to their house several times and they never come to ours. I don't like going over to their place and taking our daughter because they smoke like chimneys and I always feel horrible when we leave and I don't want my daughter around it. We went out of town for Christmas this year so we didn't go to families houses this year, my husband told them we would take her over there to get her gifts without actually telling me he set a date and time to do so and got mad at me when he asked yesterday about taking her around 7ish last night. We had things we were trying to do in town and he wanted to dropped it to go to their house, I asked him why they couldn't get in their car and drive 20 minutes to our house to see her and bring her gifts. He said it was going to be the last time we go over there because they needed to start coming over to our place more. They've seen our daughter probably 6 or 7 times in the 8 years she's been alive. All I've ever asked is they start making an effort to start coming over and spending time with her. My parents have always made sure they took the time to see her and spend time with her. She always stays with my parents because she know them well. She doesn't even know my husband's parents. My husband dad has always had the saying "the road works both ways" so why can't they make the effort?! Pissed me off and husband and I fight about it alot sometimes. I don't know what else to do because my husband doesn't see my side. He says he does but then gets mad at me when I don't want to go over there.