New job while ttc

I'm starting a new job new week but I'm really stressed out about it. For one my SO is stationed in another state (he's in nc, I'm in ct) and he brought me tickets to come down and visit him and try for our 4th cycle of ttc. The trip is at the end of this month and I'm scared I'll lose my job if I go and then again I can get the baby fever out of my mind it's all I eat, sleep and breathe at the moment. It's putting me into a slight depression when I know I should be happy that I've obtain a job. This sucks.