Feeling lonely/upset


I invited my SO over today and was hoping he would come over kind of early so we could get all/most of the day together. But now he just told me he cant come over until the afternoon around 3. He says he needs to spend time with his brother which i get, but he will also spend the next two weeks just them since they just stay home together. Usually i would have no problem with it but my parents are divorced so im only in town with him for today and tomorrow before i go back to my moms house for almost 2 weeks (the holidays are always complicated with divorced parents). Ive only been here for 4 days and most of those were spent not being able to do anything because i was sick.

I just feel like a bad person if I'm like "no can you please spend time with your brother later when im not here since you'll have almost 2 straight weeks of just you two sitting at home together?". Would that make me a bad person?

It's just that I'm an only child so when i go to my moms house im usually all alone because my mom works half the time and i dont really have any friends there since i don't go to school there.

Am i being selfish asking my SO to put off reshuffle his time like that?