Question from a novice worker-outer 😊

I recently started doing home workouts, just simple ones following different YouTube people. I don't have a lot of experience with home workouts aside from jogging.

I'm a few days in, and the muscles in my lower back are sore. It feels like mild muscle pain, not the kind of allignemnt pain that I would go to the chiropractor for. Is this something that you push through and keep working out (like when your quads are sore), or because it is my back, am I supposed to rest it?