Father In Law digging into my family without asking

I have a question and want to see other peoples opinions to see if my judgement is clouded or I have a right to be annnoyed. So my father in law has never shown any interest in me, my family, or my life in general. He doesn't know my parents names, the town I was raised in, or the college I attended. I am not the type of person he wanted his son to marry and I understand he was dissapointed. He wanted someone with his same religion and political leanings that came from a 'good family' with money. I am the 100% opposite and I get his expectations were let down. I am fine with that as he talks down to me when he does learn things about me and is very judgemental as my family is 'socially lower' than his. Its never bothered me before, it is who he is so we exist around each other politely without much communication past small talk. He has recently gotten into doing his ancestery/family tree history...to the point of obsession. He has knocked on doors of people he has never met or heard of that the has found are related to his family....some very distant relations. He got his done to the point he cant do anymore and now..without asking if its ok...is digging through my family now to do a history. There are members of my family for safety and legal reasons we have cut all contact with and we dont want to do our ancestery in general. The family we have is our family and thats all we need. He has created a ton of drama and hurt people in his family by finding out people who they thought were related werent and exposed affairs. I have asked him to stop and he refuses to do so..claiming its for his grandchild I am currently pregnant with. I am annoyed and a little angry he decided to dig into my family's history without permission and won't stop after he has been asked. I never planned to share my deep family history with my children unless they asked later in life, and some members we never wanted them to know about. He is potentially going to effect my whole family without disregard to what we want. His ancestry is importamt to him and that is fine but everyone in my immediate family doesn't want to know and I feel like it's not his history to dig into...its not his family and we didn't ask him to do this. Does this seem like a feasible way to feel or am I being annoyed over nothing.