Covid question

We got our very first house about a month ago. My husband has one family from his side and they came and saw the house. I habit three families, only one family has seen the house. We do social distancing and they came at different times. I have two families left and they are eager to come. These families got covid in November, some family members were affected and some were not. Now my husband is saying they can’t come over even though its been well over 2 weeks since anyone is these households has gotten covid. I feel like this is only directed towards my family. I don’t go out much at all due to covid. My daughter has asthma and she hasn’t gone out really either. I feel bad singling these two families out. My husband is putting this fear in me that if they come over I might or my daughter might get covid. What would you do?