I’m 28, hubby is 42...is it too late to TTC?

I’m starting to get worried. Well firstly we’ve been TTC for 4 years and have unexplained fertility. I was prescribed Clomid but terrified to take it due to side effects. My husband gets upset bc he’s getting older and I’m holding things up. I feel kinda selfish. Anyway just wondering how old was your spouse when u conceived?

ETA: I know I’m still fertile for a few more years. My question is basically....is it common for a guy who’s 42 to have baby bc he feels like he’ll be too old physically to look after a child...

#2 ETA: We’re expecting!