Does your husband make you feel desired?

Does your husband show excitement for being with you and has a strong sex drive? My husband has a low sex drive due to lower than normal testosterone. We just found this out after 3-4 years of maybe sex once a month and this past year he has hardly been able to keep an erection.
Not looking to get treatment for low T since has its been said with that it can make his sleep apnea worse.
He was diagnosed with sleep apnea 4-5 years ago and he didn't like sleeping with the mask on so he didn't wear it. I don't blame him...but going untreated has taken a toll on his body. So I finally researched and found a new machine/mask that he likes to wear. Hoping it we will see a big difference after six months. 
Yet....I feel like we are an old couple! No passion or excitement! I don't feel desired by him at all. Feels more like we are roomates or business partners. We don't sleep in the same bed due to his sleep apnea. I'm a light sleeper. So that makes us feel more like roommates.
We are in our mid thirties and we do not have kids. I fear that will really make things worse if we do. Since a lot of people's sex life dwindles with kids. I feel like I'm in prison or stuck to life sentence of a sexless marriage. 
Its more than just the act of the physical sex that I want its the desire, I want to feel that he needs me sexually. I have told him
All of these stuff he is well aware. We have been to counseling and that didn't really help either. 
There are other things besides just the sex that bothers me and that is his lack of patience and how he gets easily irritated at little things. I feel like I am constantly talking him about the need to relax and gave more fun in life.
He is very work focused and that has been is priority over the past 5 years.
All of these things combined has really taken a toll on our relationship and I for sure am always wondering if we would be happier if we went our seperate ways. We tend to argue more than we laugh. 
He tends to be controlling too. For example I am on a work trip out of state and recently one of our male coworkers moved away and is an hour from where I am working. When I tell my husband that he is going to visit me to do some touristy stuff he gets all bothered and thinks its weird. He has met him before and of course I said I have no interest in him etc. Nothing to worry about. 
Just makes me feel like maybe I would be happier single because then I wouldn't feel like I am in prison. He can be a micromanager too over little stuff and that drives me nuts! Everything has to be planned and super organized which is nice but it really sets the stage for arguements when I am more spontaneous, care free! 
For the past few months I have been doing absolutely everything I can think of to improve our marriage! I bought him the sleep apnea machine, we went to counseling, I wrote 10 things I love about him on the bathroom mirror for him to find when he came home, I have bought cards and left them in random places. I bought him lessons to meet with a personal trainer. I booked him a massage.
He reciprocated one time by buying me more personal trainer tickets with a nice card.
I hired a personal trainer and have lost 10 pounds. I grocery shop and cook more now. Which he has been enjoying! What I don't like though is how he comments about oh wow your getting smaller imagine how much smaller your going to be. Or imagine what you will look like if you keep it up! Its like he is not attracted to me at all unless I'm looking athletic etc. or least 10 pounds lighter than I was.
For the past year especially I have been daydreaming and crushing on men all over the place! Also jealous of pregnant women because of the simple fact they must have a lot of sex to get pregnant! 
I feel sexually deprived and now the big confession that just happened the other night! I've have been on a work assignment for a week as a journalist documenting firefighters! Ugh! All the strong men and lots of testosterone in the air! Of course I have been fantasizing and wonder what it would be like to be with one of them.
I went out to dinner with them and of course shots were ordered one after another and ultimately it led to a night of drinking and one if them giving me lot of attention! 
I let it happen though I allowed him to be flirty and put his hand on me... I didn't stop him because it felt so amazing to have that sexual energy that has been missing in my life for the past 5 years! 
I ended up having sex with him and the sad part is I don't feel bad about it at all! I don't feel like I betrayed my husband or anything! 
Does this mean I am done with my marriage? Should I divorce him?