Is it wrong to drink in my own home at age 20?

I'm 20. My husband is 25. We bought our own home together last year. We like to sit at home and drink port and share a cigar together when the weather is nice and it's our days off. Other times I'll drink a little wine or a vodka mixer if we're hosting friends or family.

My father hates it. We had my parents over for dinner, and we were all having a drink with dinner, and he got upset with me, asking me rhetorically "How old are you again? Oh that's right, you're 20. You shouldn't be drinking that."

Of course my husband was pissed, and told him I can drink in our home if I want, and my dad got quiet after that. We ignored his angry stares all night. Don't get me wrong, he's a great father, but for whatever reason he's so stingy about the laws. (I haven't even told him about the cigars now that the tobacco age has been raised to 21 as well)

What do you think? Is it really that wrong that I drink in my OWN home? I obviously can't go out to bars or anything, and I really don't want to since my husband and I prefer drinking at home anyway.