Boyfriend lowering phone calls when im around.

So, i came home today to my boyfriend which is basically my husband because we live together for 3 years to talking to my mom on the phone, he helps my.mom alot with doing stuff for her such as her townhome and car and stuff (my mom lives with us). Him and my mom have always been super close and always have helped eachother out with many things in life ever since he was 14 and on the streets. She basically raised him. Recently I found out they use to sleep together before we were together. For a couple of months. This was 4 years ago. I didn't find this out until a couple of months ago because it was always hidden due to me and him getting together. it was a huge fight when I found out because they both lied to me. Anyways they swore up and down and all around they never had anything after us. But I have found deleted messages between them and even nicknames like Nena, tutukita, papi and shit. I went off about that too. So I been giving them.both shit about it and whenever it's brought up they get sooooo mad about this stuff. So today I walk in and they're talking on the phone. My boyfriend's voice completely changed and hands up the phone. Talks to me about my day and everything like normal I didn't make a big deal out of it. But she calls again and I notice him playing with the volume putting it lower as soon as she was hanging up. My feeling was he was lowering it due to me not hearing the (love you) or something in the end? I brought it up to him and he got furious saying I'm jelaous and I always making stories but this is something THAT REALLY BOTHERS ME. his answer for it is because he wants too.. which isnt an answer for me. How may I bring this about to know if it's.still serving or not??