Is it me?

I have two kids 2 and 3 months. Same dad. He also has an 11 year old. Him and I have had a very rocky relationship but have been “trying” to make things work because we do love each other or have love for each other. When jt comes to his 11 year old things get weird. We used to take him everywhere before I had my two year old. Him and I were pretty close. When I got pregnant my boyfriend would not bring his son around me. After I had my son then he brought him around sometimes. (Though he only gets him 1-2 times a week.) when he does have him he is very very on top of making him comfortable and doing anything and everything he wants, which I completely understand. When his son has been around my 2 year old he has shown jealousy to the point where he trips him and has even put his hands around my sons neck. My boyfriend did NOTHING about that. Said I was crazy for saying something like that about his son. Says I hate him and he’s not comfortable bringing him around me. Oh and we do not live together because of this whole situation. He stays at my house every night until the night he gets his son, then he stays at his house with him. I have to ask him to do anything when it comes to our boys. But he will drop anything for his first. Am I wrong for feeling angry and resentful for this? He left work and dropped everything when he had to pick his first up from school when sick. My two year old is sick for the first time and he’s done nothing. My son is crying for him as he left to go to a practice for his oldest. I feel so terrible for feeling this way. I know it will never work in the long run. Hes just a completely different person when it’s just him. Then the one day comes around and he starts having panic attacks before he gets his son and then he comes home 1-2 days later acting like nothing happened. What gives?!?!