Fighting for Custody BACK STORY

I had my son at 16. His dad was 17. He was my first boyfriend and my first everything. From the get go my mom had a bad feeling about him. She was right to. I was very sheltered and naive and my single mom is a very strict Christian and didn't ever get to giving me the talk. So when my ex told me "you can't get pregnant the first time" I said well okay you took sex Ed and I didn't so you must know. Well I did get pregnant. And I did struggle with being a teen mom. I loved my son, I worked 2 jobs all while going to high school. When I had to be home tutored I took it very hard. I still wanted to be around my friends. I gained a lot of weight. I went from 113 to 172. I struggled with body image horribly. Post partum depression was miserable. Also, my ex cheated on me constantly so I was always mad and yelling at him and breaking up with him but in a day or two I would take him back. This went on for 6 years. I wasn't allowed to go anywhere but work school his house and home. I wasn't allowed to hang out with my friends or cousins and when we would hang out with his friends I wasn't allowed to talk to them bc if I did that meant I was trying to be the center of attention and he would lecture me about being an attention whore. We weren't allowed to watch the news. Only cartoons. I wasn't allowed to play Tetris bc he thought I was texting other guys. I had to play swords, wrestling, and Nerf guns and somehow I always ended up hurt bc of how rough he was reaching our son to be. When we went on a cruise with his family, he and our son shared a room with his parents and I shared a room with Jos grandmother. That's all fine and well but we were 20 and 21... we weren't allowed to share a cabin. I also wasn't allowed to do anything I wanted to do. He would ask me what I wanted to do and he would instantly say "no we aren't doing that." His parents at least offered multiple times to watch our son so he and I could do something together and he refused bc he didn't want to leave our sons side. After day 3 of our 7 day cruise I was told that I wasn't allowed to dry or style my pixi cut hair, shave my legs, or apply make up bc taking 30 minutes to get ready for the day was selfish of me. So I didn't do any of that any more. I also wasn't allowed to take longer than 10 minutes to poop bc that was selfish of me to occupy the bathroom that long if his grandma needed to go. I decided then to only speak when spoken to and just smile. My showers then consisted of me silently crying while I washed my self as fast as I could. On the last night of the cruise he finally got his parents to watch our son. He said we were going to the bow of the ship. I was kind of excited! When we got there he lectured me on how my constant silence was rude and how I had behaved very badly and was very selfish. I paid $450 to go on vacation where I wasnt allowed to do anything, buy anything, I was on time limits and restrictions... I decided then and there that I was going to leave him. It was a matter of strength. My 21 birthday was a few weeks later and he didn't allow me to have a drink when we went to dinner. So finally thanksgiving came around and I enjoyed time with his family and hugged them for the final time. I left him the next day. He didn't like that. He would try for years to use our son to get me back. It didn't work. Finally, it's 6 years after I left him and I'm getting married to a man that I met my senior year of high school (ex and I werent together most of senior year) and I had always had a thing for this guy so it just seemed meant to be. We are getting married May 20th 2017!! He and I have a 2 year old and an 8 month old together :) my 10 year old is a great big brother! We just have a hard time with him when we get him back from his dad. From the time he was born he has gone to his dads house from Thursday afternoon until Saturday afternoon/ Sunday evening, alternated. This is what needs to change. His dad lives 2 hours away going to school so his grandma picks him up Thursday after noon (she lives 45 mins away) and takes him back to school on Friday and then picks him up again and his dad comes up on Friday afternoons when he gets out of class. Until this past February we had never made any special requests to have him on the weekends bc he told us that we weren't even allowed to sign him up for sports/activities since they'd take his time from them hanging out. He then said "you can actually sign him up for as many hugs as you want but I'm not taking him to anything. This is my time." So our son hasn't done anything sport like since he was 4. He wants to but his dad has him convinced hat all he'll do is get hurt and that it would mean they wouldn't get to hang out much since he'd be playing on a field. Wtf? Well our son wanted to do a mother/son dinner so his dad let us do that and hasn't let me forget it. I took HIS time from him. Then he wanted to see monster trucks with my fiancé so we picked him up at 2 instead of 4 and he hasn't let us forget it. We made it up to him by saying since he had two short weekends in a row he could have two long weekends in a row to even it out. That isn't enough. He says that didn't make it up. It was still taking his time from him. Then we asked to have him stay with us one Friday night (on his short weekend) since we were going on vacation from Saturday til Friday. He got our son the Friday we got back at 2pm until 630pm that Sunday evening. But that was still taking his time from him. Saturday we got him at 230 so that he go with his church group to a concert. He was pissed and said that after the wedding he is getting him several long weekends in a row. I said no. He has gotten him for two weeks straight every summer to go on vacation and I never complained all while still getting his same weekend visits. We have him during the week but we don't have "free fun time" with him like he does every weekend. He doesn't make him comb his hair, he dresses him in clothes that are 2 times too big but his undies are 2 sizes too small, he lets him eat whole pizzas alone. No fruits or veggies. No chores. They share a bed at his parents house. He wouldn't let him learn to get on and off of a bike unassisted... he doesn't use manners and comes back rude and cries when we tell him no bc His dad gives him everything he asks for... so once we are married my fiancé and I want to fight for custody. We want him every other weekend. I'm so tired of being made to feel guilty about wanting to spend time with my son as a family! I've been so compliant with his dad and now that I'm standing up for myself he is trying to retaliate. What can I do to make a custody battle go smoothly? I'm afraid of him using my past against me how I used to smoke cigs, go dancing with friends when he was at his dads, and how I would go off on him when I caught him cheating. I'm just so scared of not winning more time with him. We love him and want him to be included in more family things but his dad is resisting. Advice?