I need advise please!

I'm 9 years younger than my boyfriend, we live together and are very happy. He has always said he's wanted children and in march told me to come off my pill. I was obviously over the moon only to be told in April he wanted me to go back on, so I did.. I then came back off the pill in May and have been off it with no luck of getting pregnant. As you can imagine I was devoastated when he came home this week and said we couldn't afford a baby and he wanted me back on the pill again! I feel like it's really messing with my body and my hormones as well as my feelings. Is it doing my body harm by stopping and starting the pill? And really, can anyone afford a baby?! The way I see it is you make do. Me and my boyfriend earn a lot more than my parents did and they had 3 children and managed just fine, is this really the issue or am I missing something? :( please help...