Understand that an allergy is the immune system's response to a new thing. You might acclimate to your cat. My BFF was severely allergic to cats and could not spend any time in my house. Over the years, she was able to sit with my cat on her lap and later adopted one (or 12) of her own.
Take OTC medication for your symptoms, have someone else (if you can) vacuum or sweep cat hair daily. You might be ok.