Advice pls

So my partner and I have been seeing each other since Jan. During quarantine he practically lived with me. Now I've gone back to work I barely see or speak to him. He works two jobs and can finish very late so he says he doesn't want to come over and wake me as I have to be up early. I've told him I don't mind but he still doesn't come over and only sees me on a Friday night. He believes absence makes the heart grow fonder where I'm the opposite, absence teaches me to be with you.

I feel like I'm becoming hard hearted with him, but I really love him. I just can't cope with the disappointed of him saying he's coming and not turning up, no text or call.. He has said he's scared of relationships from the beginning and now I feel like he's pushing me away although he says he's never been loved like this before and doesn't want to lose me..

He also has an issue with the 14 year age gap between us... he says he's scared that I will think he's old and leave him for someone younger...He seems to have trust issues and insecurities...

I'm in love with this man... I'm just asking what you would do, would you keep going or break it off?