1st trimester and sex

My SO does not want to have sex. He says that we should wait till the second trimester. We were ttc for 3 years I get how he's worried but it just seems irrational. Idk what is yalls opinion on it.