Weird... I've been really emotional this past couple of days, maybe that's why, in my head, I'm taking it out of proportion... But how would you take this?

My husband and I were cooking breakfast and talking about how his brother is looking for an apartment and we think it's because he wants his new girlfriend to move in with him (he's 21 and she is about to be 18, just FYI). Well, my husband and I have lived together for 3 years and not married yet (I call him my husband because here in Texas we are legally married my common law and because that's how we like to refer to each other), so I jokingly told him "watch they're gonna end up getting married before us" to which he quickly, and with a serious face, replied "that's because they don't know what they're doing".

Ok, I agree they don't know what they're doing because they haven't been dating for too long and are relatively young, but... What about us?? Do we " not know what [we] are doing either?? Does he freaken think that??? Is he doubting us?? It's been 3 years and we have two kids.. Wtf?!!!