How is the condom supposed to fit?

When my boyfriend and I have sex and he puts the condom on sometimes it only goes halfway down and then it goes all the way down later... he says it’s tight but not that tight and it won’t roll all the way down sometimes. Do we need a bigger size?