Dementia and care of a baby

Ilona • Engaged ❤ Baby boy 30/01/2017 ❤
I had a big argument with my partner because i expressed my concern leaving the baby with his stepfather who is 71 years old with dementia and his sister who 52 with Alzheimer's. 
Am I crazy? Too worried or what?!