Should I leave him alone?

I meant this guy in 2021 we was just friends with benefits. Fast forward to 2022 I get pregnant by him a few months we make it official. Six months later we both say I love you. Then I got pregnant again unfortunately I lost that baby and I when I told him that I was pregnant he said for now on we have to use protection so i agreed. Well then we got a place together because we both was inlove with each other. He even said he wants me to have a son with him in a few years. But I don’t think he really loves me because he never broke up with me at least he never told me. So for the past couple months he say he just wants a casual relationship. He said he isn’t just is not monogamous so that hurts because I am in love with this guy. So I told him we can have a casual relationship. Which I never been in a casual relationship so i asked him what is the rules and he said I want you whenever I want no restrictions. My question is do you think i should leave this guy alone and just co-parent our daughter?