LONG POST!!!

So I have been with my man for a year and a half . I had 2 children from my 1st marriage age 3 and 2 . I’m currently 11 weeks pregnant with my boyfriend. So a few months ago I moved out of state with him so we could be around his son. Well the mother of his first child will absolutely not let me or my children around her child (I know it’s here right as a mom ). So they have a “meet up” spot witch is his fathers house for supervised visits. She has randomly tried to start drama in between me and him because she wants the relationship back.. I’m very respectful about her being the mother and I keep my mouth shut. He has given off some vibes that something has happened between them since moving out here.. then I get a message saying that since we have lived here my boyfriend and his ex has been having sex ... he tells me that’s it’s all a lie and called her out on her crap.. her response “well if you would answer your phone I wouldn’t start drama “... I so desperately want to trust my man but given history between then durning our relationship makes me question a lot of things ... I just don’t want to look stupid for staying with someone who is cheating and I’m just love blind... she told me that she still loves him and they still have a connection... I am at a loss for how to handle the situation... I told my boyfriend that if he makes me look stupid i am done and I’ll be moving back to PA ... but being pregnant and my children call him dad my heart is torn ... please help ☹️