BBT and Time

I am new to charting, my husband and I have been ttc #2 for 7 months so I thought I would try something new. This is my first month and I'm finding it hard to take my temp at the same time everyday, does that really make a difference? One reason is my daughter who is 17 months wakes up within an hour range every morning and the other is my husband travels so some days I am up earlier than normal when he is gone. I am 5 dpo and my temps have been low 98's and this morning I woke up 2 hours earlier and my temp was low 96's. Is the time really making that difference or did it drop because I'm not pregnant? Any advice or tips would be greatly appreciated and good luck to everyone!