Balancing needs

Any advice on balancing the needs of me, baby and husband. I feel bad because I can tell my husband wants to have sex but I’m so worried something is going to happen cause we are only 6 weeks along. I know it’s dumb. He is kind and understands but I still feel bad cause I know he wants to. And I wish I did too. I know that as I grow I might continue to feel like this. I imagine this is more common than I think. Any advice?