Weird renting situation

So.. like the title says this is a little weird. I just moved in with my boyfriend. We live with his father who travels for almost months on end for work. Before I came to live here his father's brother used to live here with them. He put my boyfriend through a lot of abuse and my boyfriend was basically his caretaker for years. It was a lot and he is still healing emotionally from it all. I don't blame him. The "uncle" was an angry drunk who literally disabled himself with drinking. He made it so my boyfriend would not want to leave his room unless he absolutely had to. Because of that the house fell into neglect. When my boyfriend's father came home he would clean up small things, but the whole root of the problem was never addressed. Nothing fully cleaned up after. My boyfriend tried, but felt like there was no use since his uncle just kept making it worse again. My boyfriend refused to really buy himself groceries because he either had to eat right away or his uncle would eat his food even though his uncle already had food of his own. That's kind of how bad it was. Also the landlord was never any help because he would never come to fix anything that they told him was broken. The gas stove can't use two burners? Oh well. There's a hole in the ceiling because 7 years ago when it was a small little thing from rain water, the landlord refused to fix it so it grew and now he's upset over it. The dishwasher can't be used anymore? Handwash I guess.(I dont mind hand washing but it's just a little tick to the list). The refrigerator has mold in the rubber sealant? Oh well. There's a patch of mold growing outside a window but no visibility of anything on the inside. Let it grow. Since I moved into this place there has been so much cleaning and renovation. The uncle literally caused water damage to the floor in his room because he would let his leaky AC run all day in his room. Why was it leaky? He didn't clean the filter ever. So there's also paint hanging from the moisture. There's probably more that I cannot think of but.. yeah it's just a mess of an issue. A few month ago,

our landlord unlawfully entered our apartment. He knows I live here, he did this shortly after we spoke to him about me moving in. He had workers over in the basement(I'm told that's a shared space but it is divided into sections for tenants so I don't know if he's truly allowed down there without permission) all day(7:30AM-5PM). Didn't say 1 word to us about ever needing to possibly come inside of our home. He didn't even didn't even tell us this work would be happening, the lower tenant literally was sending the workers away because she hadn't been told either. We left at 3-3:30 to go and run to my boyfriend's mom's so we could get some advice about a parking ticket. About an hour later we get a phone call from his dad asking about why we let the landlord in our house and that's when we knew he just let himself in. Waited til we left and just let himself inside. He went into every single room in our home, threatened the health department. We get a note on our door after the weekend that says he is going to inspect our home for a certificate of occupancy. We have as much as we possibly can ready. The inspector comes and declares that all the repairs needed are the landlord's responsibility. Two weeks later we get a paper from our landlord with the health department letter head and such, addressed to him, saying that WE have to do all of the repairs by the 1st. He even penned in an extra thing he wanted done by then and is threatening eviction if it isn't done. We are doing it all. We have been working diligently on everything like we had been since I moved in. I figured there was neglect on both parts, it's not horrible to assume we could help with it. But today my brain keeps going back to the fact the inspector told HIM this was not for us to do. He specifically said it was the landlord's duty... so my question is basically if this is our responsibility or not? If it isn't up to us, we are actually legally allowed to deduct the supplies and such from the rent which is why I'm curious. Thanks in advance. Sorry it's so long, but also thanks if you hung in there. I also don't know exactly what group this would fit in so if it fits better elsewhere let me know!

TL;DR: Can the landlord tell us to fix up the apartment even though an employee from the health department told him it was his job?