advice...

ok so.. me and so have been together for 4 years... we live next door to his sister and her family.... he's always working from 9-4 then once him and his sisters husband get home they both go over to his sisters every night... it's like I never get to see him... he's constantly over there and leaves me home alone alllllllllll day long.... he asks me to go over but there's alot of drama history with his sister bc she's a very nosey person and we have our problems with each other so I choose not to go over...but I just feel like I'm not as important as I uses to be .... and he can be in there for hours till like 11 at night I just feel like he doesn't want to be around me