When using the Pan mouse function, the mouse often selects text/images rather than panning. There is already a Select mouse option, when using Pan, you should not be able to select text as that's not what you're attempting to do.