Am I being selfish

My SO work nights except Friday he is off, Saturday he works in the morning and Sunday he has the whole day. We really get no free time when he home days he sleep, so when our son napping so is he. Friday nights my son never wants to sleep (idk why) and he need to sleep because he go in early Saturday morning. Saturday he come home spend some time with us and most Saturdays he go out with his friends by Sunday I'm low key annoyed and I just want to prepare for whatever has to be done the week coming and deep clean our house. I really don't want him hanging out I'm home all the time with our son and most of our time together is spent handling business or with our little one. None of my friends live here and really I just want to be with him anyway, I stop working to be here for our son. I'm so depressed. I just want him to spend all his free time home with his family and maybe try to sneak us some free time. I don't have a babysitter because I don't trust strangers. My mom live here but she either working all the time or refuse to keep our baby till he one.