Most likely his mother. She was very close and overprotective to Lovecraft' and she was also super racist and went crazy because she got wind of the many black prostitutes Lovecraft's Father used to visit. Is no coincidence that Lovecraft married a Jewish woman as soon as his mom died.