Giving up Parental Rights

Yesterday my children’s father agreed to give up his parental rights. He said he can’t “deal with me” and I’m spiking his blood pressure. I do not communicate with him unless it’s for pick up and drop offs that I do or for him to contribute financially. I’ve given him grace for awhile to get back on his feet, prior to him not having a job for awhile he would say things like “I’m so jealous, I can’t wait to buy things for them the way you do” but now he has a job and no change. He started sending me $50/week for 2 toddlers and I didn’t want to be greedy so I said ok but then he stopped paying after 2 weeks. He only gets them Friday night-Sunday afternoon or sometimes Monday morning aswell, we were doing week to week coparenting but again me giving him grace due to his work schedule, I had to switch mine around so he can have weekends to help out. He keeps saying he needs to save money for another apartment and car but I let him know that his children cannot be put on hold while he’s saving & he has to help out. The other day I came to him and it turned into an argument, in all transparency I blew up for the first time in a long time because I am physically and mentally tired. I’m handling everything practically by myself and he ends up telling me to “keep the kids”. I took a day to calm down about the situation and came to ask was that his final answer. He basically said yes, I said ok I put everything in order and head to the courthouse. He then asks me can he take them out to eat one last time but it gave me an icky feeling so I’m not allowing them to go. All in all, it’s a financial issue .. that’s all. He doesn’t want to help out with his children and it “stresses” him out and I’m so hurt for my children 🫤 Any advice going forward? Court advice? Am I doing the right thing? I don’t want my kids to grow up and think I kept them from their dad