confused...🤔

Sorry it's long but I want some input?
So when me and my husband have sex he always pulls out like way early but a couple days ago while we were doing it he asked if I wanted to start trying to making a baby then because we have talked about it before and we both really want one and everything so I said yes so you know we continued and he came inside me he was all excited about it when we talked about it afterwards but I told him that it would be harder for me to get pregnant right now since I ovulated like a week ago and he seemed disappointed but then last night we did stuff again and it just went back to normal where he pulls out, did I hurt him some how by saying it would hard