dating???

so for the past month ive been hanging out with and forming a bond with a guy and last night we decided on “dating without the label and publicity” should i be worried that he doesnt want to be public?? hes always been so sweet and stuck around and worked with me through all of my problems and is even taking me home from therapy tonight but ive been hurt so many times im scared this is no different hes always been very understanding and caring though and over the course of the past month hes gotten upset with me only twice the first time being when i told him being so nice to me was almost a turn off and the second when i made a joke that insinuated he was only there for sex which ended with him upset that i would “think so low of him” and “how could i not know he actually cared about me” so theres mixed signals all over and im already not the best at this so my head is spinning pls help