Frustration and Disappointment

Me and my husband have been trying to get pregnant since March 2016. At first the doctor said it was because I wasn't ovulating every month which was causing my period to be highly irregular. Finally my periods have become regular for the last 3 months but still no baby. I'm defeated and at the point where I want to give up. I have 2 beautiful boys, so I should be happy. I just feel incomplete especially since it was so easy to get pregnant with them both, but I always pictured a girl (even though I wouldn't care if it was another boy). Is it selfish for me to feel upset?