is it okay or not okay??

so i like this guy.. but he has a daughter... but the baby mother is out of the picture but they share the kid individually.. i know this because he told me... hes 24 years old... im 22... Im scared because if i go on a date with him or if were boyfriend and girlfriend, what if the baby mama talks shit about me or whats to fight me? I always used to tell myself that i wouldnt be with someone who has a kid already but now i like this guy...😪what do i do? move on or just be friends with this dude?