Suppose to be trying

My boyfriend and I are suppose to be trying for a baby. But you can’t have a baby without sex right? Well we don’t get to see each other very often due to work so when we see each other we try to have sex. Well today he refused to have sex with me and told me it was a test to see how I would act because he doesn’t feel like he makes me happy without sex. I’m always happy with him the only time I’m upset is when I have a bad day at work. Does anyone else’s SO do this or is it just mine?