feeling guilty

I have just found out am pregnant and feeling really guilty as my bro in law can't have children hisself a feel like me and my husband have rubbed it in his face even tho we haven't told ppl yet.....