Am I wrong for getting upset that my bf is taking my money?

I’m a 26 year old female and Ive been with my 26 year old male bf for almost 8 years now. We haven’t had the best go financially and he has never been the best with holding down a job nor saving. The beginning of this year was different for me. Ive been at my job since January and he just got a job a week ago. I’ve been saving since staring work but sadly most of that has now went to bills so I’m starting from scratch. Trying to pay some people back from the times we didn’t have work or a income so we borrowed to make it by.

My boyfriend works about a hour away from where we live so gas is quite a bit compared to my 5 min drive down the road. He’s hasn’t had any money so I had to pay $260 in gas for the last two weeks which is mine and his combined to get us both to work. Until he gets paid. I’ve also let him borrow some money from me to get him his necessities and he promises to pay me back when he gets paid.

I’ve been doing the Dave Ramsey method (budget binders) to save, so I leave my binders at home when I’m away at work in the morning.

Well today I come home and my boyfriend has told me that he borrowed $20 from one of my cash envelopes and that he’ll pay me back Friday when he gets paid.

I dont know if it’s because I’ve let him borrow $80 already this week or if it’s because I’ve had a long week at work so I’m just not in the best of moods. So I got upset with him and told him that I don’t like that he didn’t even give me the choice to say “yes you can borrow the money” or “no you can not”

He just assumed since he was getting paid Friday that it would be alright because he’ll pay me back… so what I’m asking is.. am I the asshole?