Anal play

The guy I'm dating is barely interested in regular sex or even touching my vagina, but he constantly asks me for anal. I say no because I've never done it and it's scary to me. I also said I was saving it for someone serious. He still fingers me in the ass and wants to put things up there.... Which would be ok I guess if he payed some attention to my vagina too.... I'm just confused by this behavior, anyone been in the same situation ?