I feel like he forced me even though I still wanted to...

I've been tryna withdraw myself from my baby father as he's openly speaking/sleeping (not sure probably)with someone else. Last night he just showed up I was kinda happy although still angry as I haven't seen him in 3 weeks. So I let him in. He instantly starts rubbing/kissing my belly I don't know what to do, he never done none of this with our first.. I keep changing position and just get on with what I'm doing, when were sitting down and talking I tell him to leave because he says he's still chatting to her( we hadn't spoke for bout two weeks cause him ignoring me then me ignoring him) he turns me to face him with legs open and I'm like what is the point in this!? (I want him but I know I'm not good enough otherwise he would be with me) he says because we love each other and that we both know the point in this.. eventually he goes for my vagina and I don't say no, I guess I like the feeling of him wanting me.. Hes been kissing me and romantic when we sleep together which yet again he wasn't like with our first, I don't know if this is because she's changed him, he does love me or he just wanted sex... But he done all this and left within hour and half hasn't even seen our child and now I feel so shit, I can't stop crying I want marriage and family so bad I don't wanna be on my own everyday but I don't want him to keep doing this to me I seriously can't cope! Now we're slept together again I feel even worse I was finally starting to move on! I've been raped by a partner in previous relationship and I find it hard to say no too unconsciously cause I feel like I'm gonna have to anyways, I hugely regret, I feel like he's took advantage of me. I've got my scan tomorrow to find out what I'm having I'm so miserable and I doubt he'll be coming. Advice please 😔