Need help I'm in love with a married man

Ok where do I start, well I'm in a relationship me and my boyfriend started out fine now he seams like he wants nothing to do with me and a married man that I work is giving all the attention that I'm looking for he says he's happily married but if he was he wouldn't be wanting me right??? We like each other and talk all the time and we have been thinking about haveing sex. Don't get me wrong I know I shouldn't be doing this an I've never ever thought about cheating till I met him or even talked to another man while I'm in a relationship