stepdad doesn't like boyfriend help

So my boyfriend and i have been dating for 5 months, and when we hang out I usually go to his place. he hasn't been over here since Christmas eve, and when he does come over here my stepdad makes little comments and is really rude to him. he claims he's not racist but he does not support our interracial relationship, and now my boyfriend won't come over because of my stepdad. he says he doesn't feel comfortable at my house and I totally understand but I don't know what to do to fix this. I've talked to my stepdad but he doesn't see anything wrong with what he does or says. I'm afraid he won't let me see him if my bf never comes over, but I know he has no interest in coming to my house.