Fear of giving birth alone/not knowing until too late

Ok all my friends think I am mad, and I live maximum 6 minutes from the hospital but I have this growing fear of labour. Not the pain but if I am alone, i don't get to the hospital in time, i don't have contractions and the baby just falls out. 
Does anyone else have these fears, and am I really being stupid?!