It's not just you, it's part of the running gag on Patchouli calling Marisa a "mouse", and rating the guards that are supposed to be keeping her out on their "catness". She's sometimes portrayed as a catgirl herself.
I love this artist's portrayal of Patchouli, especially the Code Geass/Zero Patchouli.