Venting.. Need advice please

I hate my brother. I feel terrible saying that but he frustrates the crap out of me. I'm 24 and he's 26 
My husband and I had to move back into my parents house temporarily, not sure how much longer we will be here since we are expecting a baby in September and my husband is a disabled vet and I had to quit my job to take care of him, but will most likely be going back to work after the baby is born. Anyway my mom says I am so mean because I tell him shit straight up, I don't try to be nice about things because he is a grown man and has ZERO responsibilities. He has no job, he decided to quit by simply not showing up to work last November and recently tried to contact the manager to see if he can come back because his friend told him to. I have had a job since I turned 16, bought and paid off my own car and moved out by the time I was 19. My parents try to be strict but they baby him so much my mom claims he has ADHD but honestly I think it's just that he is treated like a child therefore he acts like one. I need advice I don't want to be so mean to him but if he does have ADD how can I help him? I've heard of a lot of successful people with ADD, I just don't understand and I hate how frustrated I get with him. I have tried looking things up online and every time I try to talk to him I get nowhere, and to top it off he is all about "conspiracies and the government being against us" every conversation I try to have with him always leads to that. I just can't..   
EDIT: this is anonymous because I feel embarrassed for feeling this way. And thank you so much ladies for your advice and comments! I really appreciate them