Feeling annoyed

I really look forward to going back to work, not because I don't love being home with my son but I miss social interaction and getting out sometimes. He's been especially fussy lately and I can only hold him sitting up or else he screams. It's been hard the past 6 weeks like any other mother knows. My husband doesn't really understand and thinks every woman wants to be a stay at home mom and I thought I did till now. To be happy and be the best I can be for him and my family I think working will help and we have great family to watch him. Now he's making me feel bad like he will need to go part time to make it work and that I shouldn't feel the way I feel. As he goes to work every week day and spends some of his Friday golfing with clients and Saturday morning with his dad.....what should I do? Accept being a stay at home mom or make it clear what I want? I'm not trying to feel like a bad mother by wanting to work.....and I shouldn't have to feel like that so please I understand why some love staying home just isn't 100% for me and I'll miss my son so so much but a few hrs a few times a week will be good.