I am wrong to feel a little upset?

Whenever I fall asleep at night my boyfriend waits until I'm asleep to masturbate. I don't really know why but this irritates me a little bit. Some days I'll tell him I'm horny after a week of not having sex and he will say he can't because he has masturbated the moment he got home from work. Does anyone have similar situations and get irritated by this as well?