8 month old goes to bed at 5:30pm

So my 8 month old likes to sleep at 5:30pm hes Exclusively bf so he sleeps with me. now during the night hes up at like 4 times but just to be put back on my boob, he then wakes up at 6-7 am .. he sleeps for 2.5 hours a day but if im coming from my granny's and he fall asleep before 5 pm he stay up till 8 and wake up ready to go at 4:30 am theres no in-between and he sucks at nap if i stay on the 5 pm bed time he'll take 2, 30 mins nap and then a hour nap some times hell take a 15 min nap , but with the 8 pm bedtime he'll take he'll sleep the 3 hours during the morning and wil be pissy till 5 pm ... but everybody says 5 pm is to early of a bedtime so i was seeing if any body can crack the code for me cuz ive tried to put him on a schedule since 4-5 months and he throws me for a loop everytime .