Im getting mad at myself..

Soo me and my boyfriend are together for an year and a half and we have had sex alot of times but lately I dont realy feel like doing it . I can feel that he really wants it but I feel bad for saying no to him over and over again .

I think that it all started after he got close to hes buddies about two months ago and we didnt meet and spend as much time together as we used to . Im kind of getting jealous about him not paying as much attention to me anymore . Also, we do meet almost every day, but he always comes to my place at 10-11pm and he always wants sex! What should I do? I am getting mad at myself because we used to have alot of sex, but since he is mostly away, I dont want to do it anymore . Help .