pissed at my boyfriend & don't know what to do

Janey
(warning: kinda long) I'm not a petty person, but my boyfriend pissed me off and my whole family is already mad at me so it's making me angrier.  I'll spare the run down of what's going on with my family at the moment, but the only important part is I didn't really do anything and my brother wants me to delete his number and not talk to him ever again.
 I'm away at school right now and I go home on the weekends and earlier in the day I made a stupid joke that was like "you haven't seen me in three days you probably got married had kids got a new job and forgot a what I looked like" but I was kidding. I guess at some point I wore it out and it stopped being funny and he was like "ok I'm tired of repeating myself about the same thing" as well as something else that I don't remember. but I told him I was just kidding (which I figured was obvious because I didn't expect him to forget what I looked like in the course of three days) and he just goes on to say when you keep the joke up it wears out and it seems like you actually feel that way and jokes have some truth behind them, I wish you would stop saying I've forgotten about you when I haven't. so I texted him back "my bad" and he said it was all good but I could tell he was still pissy about it and I said was ok and I know he's not going to respond to it or start a new conversation at any point.
 he's really bad at communicating over the phone. not just with me but with everyone. so while I'm away at school literally days will go by before he's like oh I haven't talked to my girlfriend in three days I should probably call and check on her. so I know that's going to be the end of the conversation until I see him again (or I start one but I feel like no one wants to talk to me so I'm not doing that)
I started to say "if that was the end of the conversation & we're not going to talk again until I come home love you, guess I'll just see you Friday" but I hesitated before I sent it because I feel like if I say that it's just going to put him in a pissy mood while he's at work which no one needs to be put into a pissy mood while they're at work, and that will drag on until Friday and maybe even longer. and he didn't really do anything for me to be that mad at, but its just the fact that my family is already mad at me when I didn't do anything so to have him get pissy about something so dumb when I know he's probably not going to talk to me until I see him I'm annoyed. but as annoyed as I am I still don't know if I can go out of my way just to piss him off. but i feel like I have the right to be pissed off when my family is already not talking to me so for you to be the one i should b going to at the moment for you to just be someone else who's not going to talk to me because of somethings stupid, I feel like too bad if you're mad, I deserve to be mad you're making things worse for me, and I don't need you to act like a little girl when I'm already in an unhappy situation. 
I don't know if I should just get over it, say something, just wait until I see him over the weekend if I even decide to go home