Still in love with my first . Good advice please ladies!

I have been in a realtionship for 4 years with a wonderful man . but i still find myself thinking about the guy that took my virginity when i was 16. He has a girlfriend now and two kids (not by her). And when we see each other we smile super hard. But we never worked out back then because he treated my horribly. Should i tell him ? Or just forget about it and stay with my boyfriend ! Need advice !