Is it wrong I'm annoyed?

DH and I have been trying for 6 months (not that long I know, but it feels like an eternity) and I just feel so empty because I haven't gotten my BFP yet. Monday we went to his step dads house for brunch and I got a terrible migraine. So, naturally I was nauseous and dizzy. I went to go lay down in a spare bedroom to be away from noise and I overheard all of his family make comments about me being pregnant. "Is she pregnant?" "Isn't it too late in the day for morning sickness?" Etc. It annoyed me to no end. Yesterday, our teenage nephew asked if I "had anything special to announce" He wasn't at brunch so I'm assuming his step dad (my husbands brother) said something to him about me being sick. It just annoys me because I feel like it's so insensitive. They know we want kids, and know we've been struggling since we've been married for almost a year with no baby yet. (His family all knew we wanted kids right away)