Is it my fault?

So I was on Bc pill not anymore. Anyways, I was having a intimate relationship with this doctor and I told him he can cum in me not to worry because I’m on (mini pill) and he said sure I will if you’re on the pill and fast forward yesterday I wasn’t feeling right took a test and I’m pregnant. I thought I was religiously taking them turns out I missed my two pills the day before we were intimate and the next day. I was sooo busy those days I didn’t remember . Is it all my fault?!?😫 I’m scared to tell him I haven’t yet I feel like he would blame everything on me.