Boyfriend's Best Girl Friend

Maybe I'm just being insecure, but how would you feel if your significant other lives with a female housemate whom he has many classes with, interacts often with, plans to do fun outdoor sports by themselves (while you're often not invited) when he's less inclined to do other outdoor activities with you that you can actually participate in, and defends her more often than he defends you? While I'm over frequently and decent friends with the girl, I can't help but feel like I'm in second, like he has more fun with her, and that I almost always have to be the one to initiate plans with him and to live with him in the future.