How often should he see his son ?

My child’s father stops by my house 3 or 4 times a week to see our 1 year old. He typically stays for 15-30 minutes. And rarely he’ll stay for more than an hour. He usually comes by after work because he works nights and gets off in the morning. I’m pregnant with our second son and due next month. I’m in school ( online classes full time) and I work four hours a week and he watches our son or his family will since they live together. The thing is I’m getting tired and I was thinking that he should have our son come see him instead of him coming here for 15+ minutes. Like maybe he could go to his house for a few hours. Or is that asking too much? I’m not sure how this co parenting thing works . But I don’t know what to do because he does work nights and gets off around 8:30am and he sleeps until around 2 or 3 sometimes .