Should we swaddle babies everytime ?


My LO is of 3weeks and i keep him swaddled almost all the time, but at times I have seen he really enjoys when i put him mittens and set him handsfree without any swaddling.

I am a FTM so really confused, should I swaddle my baby only when he is sleepy? Of everytime? Or sometimes?