Vitex while pregnant

I just found out I am pregnant today and I have been taking 2 tablets of vitex every morning. Should I wean off, if so when? or should I just stop cold turkey? I am 5 weeks.