Stopping the "pill" question

I am married and have been for 16 years. My husband and I have 2 kids. 10 and 13. He had a vasectomy 2 years ago. I have stayed on the pill since I have not had any issues with it. Usually no periods, moods seem regulated etc... I have been on the pill since I was 15, I am now 37 the only time I was off was during my pregnancies. I just feel like I should let my body do what it was me to do. I was supposed to restart my pill pack last night. I am so nervous that my periods will be heavy and crampy, or my moods will be crazy. The cons of the pill is no sex drive and it is hurting my marriage. I was taking Loloestrin which is a pretty low dose. I wanted to see what others experiences were when stopping the pill?