What should I do?

So I had sex today, we used a condom but, he fingered me after he jerked off but he only wiped his cum off his hand and didn’t wash it off (I’m not sure if the sperm is still active or if it’s still possible for pregnancy) before he did anymore I made him was his hand. Later on at some point during it we realized the condom was placed on wrongly (upside down??) although it did not seem like anything leaked out.

I know it’s sounds stupid but neither of us has any experience in this aspect, so we’re not sure if we should be worried of pregnancy or if we’re blowing things out of proportion.

What steps should I take?