He keeps waking me up by jacking off

In the morning my bf will lay there awake and rub his dick and jack off shaking the whole bed and I’ll literally take his hand and put it somewhere else because it wakes me up and I’m trying to sleep. Like today I still had two more hours till my alarm would go off for work. It also pisses me off because he hardly ever wants to have sex with me. But I don’t think he ever actually comes he just plays with himself when he does this. When I bring it up to him he says he was sleeping and didn’t know he did that but I know that’s a lie because I saw him check the time on his phone while he’s done this. So do I just live with this annoyance or what?