Should we break up?

Jay
My boyfriend and I have been together for almost 3 years now and I've always thought he was really attractive and cute and stuff, but I realized that I don't find him that physically attractive anymore. I know he finds me attractive because he calls me "beautiful", "gorgeous", etc. so I feel really guilty for not feeling the same towards him.
I've also recently become friends with one of his friends, who I find very attractive (physically and personality-wise) and I feel horrible about it. 
We've been fighting a LOT lately, but I don't wanna break up with him because I love him more than anything. We've been fighting mostly about how he spends most of his free time with his friends and almost none with me, and about how I feel like I'm not important to him. I've tried talking to him about spending more time together for a few weeks, but he's just now putting forth an effort.
I'm really confused and frustrated about this whole thing, so if anyone has any advice, that would be great!