Feel like I’m being taken advantage of

I can’t find a way to communicate my current situation with my boyfriend . We moved in a few months ago we moved closer to his kids, my daughter will start school soon with them and I work from home . He’s been asking for favors for me to watch his kids during my work hrs for example last week 3 days this Tuesday his kids mom text me on my way and he never asked me if I was okay with it and Wednesday he asked if I’d be ok to watch them another 3 days that week and I’m sure he saw I was annoyed but I said yes . Later he says nvm he’s kids mom found someone for 2 of the days idk if it was because he saw I was annoyed. I’m annoyed because they think I’m on their time, his son mention to me how his mom has planned two days out of the school days for me to watch them and I never agreed and my bf has not had a conversation with me about it and I saw my bf’s text messages where she send dirty laundry for his boys and she said “ have your gf wash it “ . I haven’t mention it to him but I feel like I’m coming off as if his kids are the problem but that’s not that case I love them but I have 1 daughter and she’s use to the routine with me working from home another 2 and that throws things off for me . One likes to eat one thing the other doesn’t like it and so on so it’s so hard . Also, she isn’t responsible . She says she’ll pick them up at a certain time and something always comes up and we actually caught her jogging on of the days when supposedly she had “ plumbing issues “ . I want to express to him it’s not his kids because I love them, I just don’t know how. I get she needs to for herself and she gets time on her weekends when she alternates, I’m a full time mom also and I get my breaks on my alternating weekends and I’m responsible I pick up my daughter on Sundays on time at 6 pm from her dad . She sometimes pick them up on Sunday and when she does my bf has to text and text to get a respond because she’s out with her friends . On weekend we have the kids I push my bf to do fun activities we go out as a family have lots of fun, I cook for all of from morning to dinner and I enjoy it but maybe im a litttle too nice and it’s easy for people to take advantage of me