Pain and bleeding during sex

My boyfriend and I don’t get to have sex very often but whenever we do it hurts a lot at first and then it’s fine, and when we’re done there’s always a little bit of blood on his dick. Should I be worried about this?