Thinking of ending my relationship

Bit of background :

My partner suffers from mental health problems and when we first got together he was messaging his ex trying to get back together we've spoken about it and he said it was purely because he missed his 4 year old step kids he has brought up since day 1 (she banned contact between them) and he even had to be sectioned after this because he tried to take his own life so I do believe his reasoning for the messages (this was 2 years ago) .

Since then he hid that his financial situation is bad and got into more debt trying to make out he was ok. Since then we've sorted through our finances and I have taken over a lot whilst he sorts it out and is able to help in a way that won't affect his finances even worse.

He is amazing helping around the house and with both my children and our joint child.

Well we had a fall out publicly in front of his friends and now our relationship is just not right his friends have ignored my apology so clearly if I stay in this relationship I'm now an outcast in his group of friends making future arrangements awkward.

The main reason I have for debating ending things is I honestly feel like since day 1 he's just never been truly happy with me I see all these images of him from before me and he's constantly smiling always with friends and doing things. With me he never wants to see friends and every picture whether taken with him knowing or taken with him in the background he just never smiles he constantly looks sad and I don't want to carry on making him miserable and not being able to make him feel happy