Visitation schedule and kids


So my husband and I agreed upon divorcing that I would stay in the house for me to stay here so our kids can go to school here. My only issue is that isn’t exactly an issue (but my mom thinks it will never work long term and it may not but it’s a solution for right now) is that he said he can’t afford to rent/buy another place with paying child support. So he was planning on renting a room right now with a friend and during visitation he visit the kids at my house while I’m at work with the exception of Saturday’s from 12-7 when I would technically be home. He also will have access to his shed (which I never go in anyway) for his car stuff he would just have to let me know when he is stopping by to grab something. My mom thinks that he is just going to show up when he pleases which he is claiming he will not. He will let me know before hand. We also have it written in the divorce papers he cannot show up without notice. Am I crazy for doing this arrangement? I’m just trying to do what’s best at this time for the kids - the least amount of chaos in their life. And since I do not have enough money to move to another place at this time. Has anyone does this amicably?