when can you quit birth control

When I first started birth control I was on it for a while but just quit taking it because I went to another country and the time zone was a big difference. I started back up on it not to long ago but I think it may have made a big difference on my sex drive and I'm scared my hormones are out of wack. Are you supposed to go to your doctor before quitting birth control?