I need your opinion ladies

so my SO has a son who just turned 7 yesterday which is why we took him to Go Bananas and spent the whole afternoon with him. The kids mother is cutting a cake for him which she invited my SO to with my girls btw we have twin girls. and spesidically said just them not me. before I wouldn't mind but now we live together we have a family and well we spent the kids bday with him and we plan on cutting him a cake our self on Sunday. I don't want to be the controlling type but I feel like this girl is doing all this to mess with me... I mean she has been trying to have us break up. It bothers me that he wants to go. should I really make this a big deal or should i just let it go.. and let him go?