LO not wanting to nurse during the day?

For the past week or so my 10.5week old doesn't want to nurse during the day.  He gets both formula and breast (both by nursing and bottle) so it's not an issue of him going hungry. He nurses when in the middle of the night and when we are In bed waking up after my husband leaves for work but after we go downstairs he doesn't want it anymore. I try to, but he gets frustrated and won't stay latched and with unlatch and relatching a million times.  So I'm left full of milk that's let down and I have to make a bottle, feed him and get him to where I can put him down so I can pump.