。。。もう、意地悪な御方。
...mou, ijiwaru no okata.
...sheesh, you are a bully sir.

Retaining respectful speech "okata" and the specific exclamation "mou" indicate that the connotation is closer to "sigh, it can't be helped :)" rather than being actually fed up with his bs.

Couldn't come up with a better translation, but felt that the existing line "...you, sir, are the worst" wasn't quite right.