Brotherly Love

Is it wrong to get with an exes brother even if you have been split up for years and we're best friends with the brother? I would never do it straight away but if it's been 4 + years and the relationship wasn't exactly the most serious one I've had and they d moved on too ages ago , am I still treading in rough water ? Or is all fair in love and war?