Hormones?

My little guy is three weeks old today (what a big boy :-) ). I felt super hormonal, crying over dumb stuff for several days afterward, then I finally felt leveled out. Then wham, a few days ago I start with mood swings, anxiousness, struggling with silly fears. 

It comes and goes, definitely not constant and it's not deep and dark like depression. I think lots of it may just be how much people/doctors say to watch out for symptoms of PPD or Postpartum Psychosis. It's more of a fear of the "what if I DO have thoughts about wanting to hurt baby!?!?" Which in turn becomes thoughts about how horrible that would be, and triggers anxiety. 

Just to be clear, I would never hurt my  kids. All this stuff just freaks me out and I'm really trying not to obsess every time I feel a little sad or overwhelmed. 

Anyone else struggling? Am I the only one who feels like a bit of a nut right now? When does this end!? I didn't experience any of this with my first!