DH worried about having sex?!

Ok so ever since iv been pregnant DH always worries about having sex and that he'll hurt the baby or that maybe "the baby will know". He works away, only a week at a time then a week home, so it drives me crazy as it is when his away, let alone when his home and nothing happens :(. Even if we do, its all over as quickly as it started.. I also feel forgotten about as im more than happy to pleasure him.. Yet nothing happens for me... No foreplay nothing, he just jumps straight in :/. I think its more frustrating than anything, iv explained to him that its completly fine and bub wont know whats going on lol but still nothing.....