Sex when not on the pill.

How many of you had sex for the first time when you where not on the pill, me and my boyfriend want to have sex for the first time together he is not a V but I am. I was just wondering if it's okay to have Sex when your not on the pill but use a condom? 
Any advice would be great.