Should I change doctors?

Ok so my OB has delivered multiple babies within my family but this will be my first baby. I did my research early on and decided not to get the flu shot based on research and my religious convictions. Well my doctor basically made the comment a month ago that I should get one if I want to be a good mom! And now yesterday he basically called me crazy and told me my baby may die because of my ignorance. I've never been vaccinated so I've had this issue in the past with people but not only was it a religious choose not to get vaccinated but also the last time i tried to get a vaccine I had to have surgery to remove a infected cyst out of my left breast at 17 years old. My husband also chimes in with the doctor and says I should get one which doesn't help. I feel like they are forcing me to get one.