Should we be discouraged??

My SO and I have been ttc for about 4 cycles now and we're both starting to get frustrated that it's not happening for us. We keep hearing about other couples who get pregnant without having to try much. We're both starting to get discouraged about the whole situation.  Should we be discouraged?