How should I feel?

Well to start off I'm almost 20 weeks pregnant. This will be my 4th child but 2nd with my current bf. Unfortunately with my pregnancies my sexdrive sky rockets. In the first trimester my bf was pretty much all over me all the time which I really liked but as of about a month and a half ago he doesn't want to touch me. Barely even gives me a kiss and doesn't hug me at all. And sex doesn't happen unless I bug him about it which isn't often. With our first child he did the same thing but it was because I almost misscarried at 7 weeks. But he grew very distant. He's starting to do the same thing and it makes me feel like I've done something wrong to him. Last night I asked him why he doesn't want to do anything with me and he said me being pregnant just makes him lose his sexdrive which makes me feel like he doesn't find me attractive. I told him he did the same thing with our son and he told me that it's probably because it just makes his life that much harder. We broke up last year in Nov because he said he was bored. But when I started talking to someone else he told me he realized what he was losing and he wanted to work things out and then I ended up pregnant which he seemed fine with at first because it bothered the guy I had been talking to. And everything was going good until a month and a half ago. Idk how I'm supposed to take what he said. I just feel like he's basically going to not want to be with me again because he's going to get bored. He says him being with me should tell me what he wants but the way he acts says he is only with me because of the kids. I don't want him to be miserable with me. He doesn't want to marry me but he says he loves me. And he doesn't want to touch me.....how do I take that? I feel rejected...😭