The hair is fine as it's not a cause of allergy's, it's actually proteins in the cat's saliva, urine, and dander that can trigger it.
Mostly the saliva and dander. The urine is only a problem for indoors cat due to their potty medium usually comprised of porous material that both allows absorbtion and retention of cat urine but also allows for creation of thin layer of urine crust, which then gets crushed and mixed with the air as fine particles every time the cat stirred their toilet.