Need advice--UPDATE

There's something I can't get out of my head. So about a week and a half ago, I had sex with this guy. We used a condom, but afterwards he asked me what if I get pregnant. I can't get the thought that he might have tampered with the condom. It didn't seem like it tore or ripped. It did feel like it moved a little when we had sex. My period is not due until a few more days and I'm trying not to stress about it. It just felt like an odd thing to ask. And he didn't really look that happy with my reply. I told him I didn't want to add more on to him but if it happens we would deal with it then. He has a lot going on in his life. It just seemed like he want me to say that I wanted to be pregnant with his baby. He looked too hopeful when he asked.

@Mandy: actually he has a high stress job and is battling depression. Sometimes the combination of the two will lead him to drink. I've known him for over a year and at first when we were in a relationship every was good and we did talk about having a baby, but I wanted to wait until we were together longer. But then he said he wasn't ready for a relationship, so we've been trying to be friends. But for a while he keeps pulling me close to him still telling me he wants me, then he pushes me away. I'm not the only one he does this to, he does it to his family as well. He's even said he's fine with me seeing other guys, but if I have a date he tries to make me cancel. I still love him, but I'm ready just to be done. This one thing keeps nagging at me though

UPDATE: We texted earlier. I told him I had something I wanted to ask him. At that point he had already asked if we could get together but I was already running errands and I knew we would end up having sex and I wasn't sure that was a good idea. So when I told him I had something to ask him, I also said I didn't want to exactly ask via text or over the phone. I was getting scared to ask him. Below is how it went

I sent a couple more text but he stopped replying. I know he got upset. I know if I try calling he'll send me to voice mail and I can't go to his place (his landlady will make a big deal about it). I know in a couple of days he'll text me like nothing happened. I just don't know how to deal with him