anal sex

So my bf really likes to have anal sex and we have done it like two or three times only, but the thing is that I sometimes feel like its too painful and not worth it, we have tried using lube but it still hurts. The first time we had anal was amazing and I really liked it but the next two times weren't that good. Any advice on it?