boyfriend and job

So I recently just got this job as an intern in an office at the same factory as my boyfriend. Except he works in the plant (like out in the shop) and I work in the office. It's really good Pay and benefits but my boyfriend is super unhappy that I'm surrounded by male colleagues. May I add that these men are older than me by 20 years + and most of the time I stay in my office. I understand his worries and his fears and I have explained this to him. I would never cheat on my boyfriend, nor do I talk to these men anymore than I have to. I hate that my boyfriend feels this way. May I also add that I've never cheated in the relationship (we've been together for a little over 2 years and live together) and he knows this, he just has extremely bad trust issues. (He had a horrible child hood) I really want to leave my job to make my boyfriend happy but the money is so good. I could really use some advice.