If the guy is having stray thoughts then it's either.

A. Because your doing something wrong or lacking something regarding your activities behind closed doors.

B. Your the one having the wrong type of thoughts and really need to have more faith in your hubby.

C. The guy is a complete douche and horn dog at which case you loop back to one until you realise it and just leave him.