Need advice

My fiancé and I have been TTC for about 4 years with no luck recently one of his friends girlfriend got pregnant and he always hang around their house about two nights ago we were talking about her experience being pregnant and their was a clear look of disappointment on his face. I had to cut the convo I made up an excuse to leave the room just so I wouldn't feel guilty for not being able to conceive now I think he maybe crushing on the girl or I'm I just paranoid? BTW as I'm typing this message he is at their house