Dealing with the ex wife and kids of the husband

Hi girls, I wanted to share my story and see if someone finds in a similar situation. I felt in love with a married man, with 2 kids 6 and 1 year old. He felt to and decided to leave his wife for me. We got married last year. I tough that it wouldn't be that difficult as we live in different countries (my hubby and I and his ex and kids) but it is very painful for me to see that he calls them everyday, he gives more money than what was agreed on the divorce agreement, and in general I feel always anxious and jelouse. I am 25 no kids or previous marriages and he is 32. Do you feel this feeling will go away or should end this for good? Also we are TTC but I feel that might be also so to compete to the other children ;( I feel very bad and confused pls help. I don't consider myself a bad person just I might have took wrong decisions