Advice needed!

Kassandra • Im 20 yrs old a mother of a beautiful 4 month old baby girl. I have found someone special that loves me for who i am and that loves my daughter as if she were his own.
Me and my SO have been dating awhile now. And i couldnt be happier to have found someone that loves me and my 5 month old daughter. (Its not his kid, just in case yall wonder). He has been talking about wanting to move back to his hometown with me and my daughter and that he wants to get married and start our own family. But what i dont get is he also always ends up saying that he thinks im gonna leave him amd change my mind about going with him. It drives me insane that he thinks like that. I dont get why he does it. I always reassure him that i wont back-out and that i wanna spend my life with him; cause lets face it there are very few guys that are willing to be with a women that already has a kid. But he continues to do it. Am i over-reacting about this or could it just be a guy thing? Please i need advice!