It hurts when I have sex?

I lost my virginity a month ago (to the day, actually!) and I knew it would hurt and I would bleed because of nerves and because my hymen getting torn, but it hurt WAY worse than I expected. To the point where I told my boyfriend to stop because the pain was more than I could handle. I've had sex two other times since then and the pain is still there, and I've still bled but only minimally. Does this have anything to do with the fact that he also had trouble staying hard? Is it just because I'm not used to sex yet, or is there something deeper I should be worrying about?