Boyfriend is atheist

Recently wanted to go to church with a man I started dating for four months and he told me he was atheist. I am very Christian and go to church, bible study all my life. I have asked for him to go to church with me but he is silent. I only want to date men who are marriage material- is it wrong to dump a man over religion? What should I say?