Low Sex drive

So me and the hubs are trying to have a baby. I've been doing my part by wearing sexy lingerie to bed and keeping things spicy. The problem is when the hubs comes to bed he looks like he just came off the basketball court; a ratty old T-shirt and a pair of shorts. I've asked him to change it up a little to turn up the heat but he refuses. I've explained to him that when I wear lingerie it turns him on but when he wears the old gym shorts to bed it turns me off. Any suggestions?