Honestly just want to know if I am wrong for being mad

I am married and my husband and I just got our first house about a month ago. He shares custody of his kid with the mom and for the most part they get along. She used to start a ton of drama with me on a daily basis and did a lot to try and break up. Eventually she stopped and we pretty much just don't talk or anything. I never get upset about them talking or meeting up to drop off their child or anything like that because i'm not immature (at least I don't think I am lol) and understand it's what has to be done. However, I became pretty upset last night when I found out my husbands ex came over one day recently while I was at work so their daughter could show her her new room. That part didn't really piss me off until I found out she was hanging out inside our house for 15 to 20 minutes. How long she was inside and the fact that my husband never told me is what upset me. I only found out because I was thinking about it and decided to ask if she has been inside yet. I guess if me and her didn't have such a bad past maybe I wouldn't have cared so much. It's also because my husband gets super jealous of my sons dad and I know if the situation was reversed he too would be mad. He acted like I was crazy and said he didn't know he had to make a memo every time she came over. His response mixed with everything else just pissed me off and I guess I just wanna know if i'm being irrational. Sorry it's so long. Opinions please!