don't know what to do.

I split up with my husband in January we've been together 7 years married for two, due to him walking out of his job it was the 8th he'd had in 7 month's or helping at home and I couldn't take the stress anymore I'd had enough we have two girls together so see him regularly and lately we've been spending time together and things have been good between us he won't be moving back in but wants a relationship I am enjoying spending time with him it's like when we were first together I'd like to try and make things work I don't want to end our marriage but my friends and family don't like him and won't be happy if we get back together I feel torn and just don't know what to do.