torn on what to do

My husband and have together for 4 years ( known each other way longer). I am a stay at home mom and I love it, we live 45 minutes from the nearest walmart or any type of stores. I only go to town once a week, he works M-Sat. I stay home the rest of the time all day long, on Sundays his days off we do whatever he wants which is ok, but what i cant handle is he is always going out with the guys and I stay home, Ive gone out once in the past 2 years, than when goes out it frustrates me and I tell him it does and he always turns it on me.. I just don't know what to do... He always says i trap him from not doing what he likes but whenever i want to do something I enjoy he says I can but i have to take Addison (our 2 year old daughter who i love she is my world but veimg a stay at home mom you need a break every once in awhile) but its ok for him to never take her or when he is home he sits on his phone i ask he him to play with her and all he says is that he is tired and that I dont do anything all day you can...