Masturbation...

I have been with my boyfriend for over three years now. Maturbation is a natural thing and it is SO healthy to do. But I feel like it is something to be done ALONE. I switched from the birth control method with the thing in my arm to the pill about 1 1/2 ago and I lost all of my sex drive! I hated sex! And I would still have sex with him but it was like every two weeks instead of every other day like before. During that time he would masturbate.... IN BED WITH ME. I hated it, I told him to stop doing it in bed next to me. I mean he was SHAKING the whole bed!!! It drove me crazy, after enough times of me flipping out about it he stopped. I stopped taking the pill two months ago because I wanted our sex life back. We have had crazy awesome sex and I have been pleasing him and he has been pleasing me too... A LOT (; .
My issue is that he keeps masturbating. Which is FINE! It is good to be in touch with yourself. I masturbate too but it's like he wants me to catch him.... He is doing it every single day and it makes me so mad because then we can't have sex because he won't get hard. Now I am sexually frustrated but he doesn't get it.... I WANT HIM EVERY DAY! Well my point is, he has never walked in on me masturbating but I catch him doing it all the fucking time. It makes me uncomfortable, like it makes me feel insecure like I am not good enough for him. How can I make it clear that I DO NOT want to see him doing that? I just want to have sex.... And jeeeeeeezzzzz.... I want to have sexxxxxx! 
Am I just horny and sexually frustrated or is this an issue most people have?
BTW I am 20 and he is 23 we live together.