Relationship advice

This might be long but I need to know if I was in the wrong at all.

So 2 different issues...My partner & I have been together 7 months & live together. The first night we had met at a bar & talked for almost 5 hours (both recently divorced). I told him how I just came out of an abusive marriage & after 7 1/2 years I cheated on my ex, left him 6 months later. He confessed to me that tho his marriage wasn't abusive or toxic, it was lifeless & loveless & he (kinda) cheated on his ex too. Fast forward a couple months later & it gets brought up again..me cheating on my ex. He wanted more details so I told him it was with 6 people over 2 months & my ex knew about it. I'm not proud of it but it is what it is. So we had that discussion & moved on. Well last night it gets brought up by him again! I tell him the same thing then he starts yelling, calling me a liar claiming every time we talked about it before I said it was only 1 time (uh, no...) & he kinda made me feel like he didn't wanna be with cuz of it even tho he knew from the beginning.

Side note- He is the kind of guy who will tell you something then when he tells the same story the next time, details change. IDK if he's a liar or just kinda forgets some things (he does seem forgetful about every day stuff) but one example is when we got together he told me he had only slept with 5 other women. Okay, idc. After living together for a month or so I asked him again how many girls & he said 5...2 months ago we were having a convo & he said something about over 100! I freaked out & asked if he'd really sleet with that many chicks & he said yeah. So how does it go from 5 to 100? That was a different discussion but he told me he had an STD test right before he met his ex wife. Then later on told me he never had a test...that was concerning with his high body count (yes I made him go get tested). I told him if he was honest in the beginning we could've got tested together & avoided all that. He made it seem like I was overreacting & being cruel for incenuating that he's not honest.

Moving on to the other issue, he has a huge problem with feeling sorry for himself. This morning he told me he's afraid I'll break up with him cuz I have a job & he doesn't. Idc about that cuz I have a lot of medical background & he only has retail management experience & where we live the job market sucks. Plus he takes care of my son when I'm working & does a great job. But he just kept going on about how he feels like he's not enough & doesn't do enough. I told him that I can't change how he feels. That he needs to stop feeling sorry for himself cuz he's the only one who change what he doesn't like. If he feels like he doesn't do enough during the day to get up & do something besides cooking & cleaning. Involve my son in something outside, something! He got all butt hurt & told me to just stop so I did. He then got up & started frantically cleaning & doing yard work... I'm not his mother & I'm not gonna tell him what he should or shouldn't, can or can't be doing. (He has been trying really hard to get a job!) I'm just not sure if maybe I'm too hard on him or if I'm justified in what I say. On top of that how do I know if he tells white lies to avoid the whole truth or really is forgetful a lot? These are really the only issues we have in our relationship but the discussions turn into fights. I guess I'm ranting more than anything...

EDIT: Sorry this is so long! My bf is super sweet, loving, kind, great with my son, does everything for me