So annoyed!!!

Ever since I found out I am pregnant (I found out 2 weeks ago btw) my SO won't have sex with me. Like what the hell is wrong with him. I'm just 5 weeks. My baby is just an embryo right now just little tinny cells. Like I could understand his fear when I'm actually showing. Why don't men understand that no matter how big they think their little friend downstairs are, it would not be big enough to touch the baby? Is anyone else dealing with this?