Need advice!!

Okaay Ive been with my So for two years already he have a 9 month old baby but I also have a almost 5 year old daughter

First thing first before we got together we had an agreement my daughter would go over every other day I would go for her at 9pm

When we first moved in together she would still go every other day she was involved with her dad we wouldn’t be to far away from each other than I got to every 3rd day or every 4th day it fast forward 💨

First it was that it wasn’t right for me to go pick her up at his house he lives with his mom or to take her all the way to his door

Took me about 2 months or more to stop doing it I would leave her at my moms and he’s mom would go pick her up and drop her of there than I told her I was already there and Shel come outside and get her right

So now it’s the you don’t have to take her or pick her up they have to come pick her up and bring her home

They don’t have days like every weekend I’m coming for her it’s when ever they want to come for her

I honestly don’t know what to do my so want to spend more time with me

But we’re always depending on weather or not there coming for her or not

To make it clear my daughter loves my so she even says she has 2 dad he’s also good with her we take her out to have fun and everything