Missed 2 days and had to wait to start pill till Sunday

Hey, so I am a pill user and my boyfriend and I had sex during my period last month (condoms each round) and I missed taking my pill for two days. I decided to just start on my new one on (3 days between) Sunday. I am 4 days late  and I'm starting to get extra stressed about being pregnant (I have many health issues and child birth could essentially end my life (I try to be extreamly careful)) should I be worried about pregnancy? Am I going nuts, help..