首筋に触れてみたけれどいまいち反応が鈍かった。
ふと、ふよふよ浮いていた半霊をどかそうとしてみたところ

「あ、だめです！」

突然慌てだした。どうやら、半霊に触られると困ることがありそうだ…。

妖夢がきもちいいと言っていたところ＞post #1379989

Had to check my grammar dictionary to be sure, but verbs that take an indirect object via に continue to do so even when they are conjugated in the られる form (which marks the agent with に); that is, the に here is performing the same function as it does in the first sentence's 首筋に触れて, and the に that goes with the られる conjugation (presumably 人に or something) is treated as implicit and dropped. Thus:

I touched the nape of her neck, but her reaction was pretty weak. But when I happened to push her floating ghost half aside...

"Ah, don't do that!"

...she suddenly got flustered. It seems that being touched on her ghost half bothers her...

Notice that this makes a lot more sense.

(Bonus fact: in cases where both indirect object and agent are explicitly specified, the agent must come first; for instance, the full statement here would be 人に半霊に触られると困る.)