I'm allergic and one cat was just a mild nuisance. Two cats was a a pain but nothing unbearable. Third cat brought serious sneezing, sniffling, snotting, and itchy runny eyes. There are some basic things that help. All surfaces they lay on (couches, bed, chairs, etc) all have a colorful blanket/sheet thrown over it that's easy to wash. I change them twice a week and watch the 'dirty' ones IMMEDIATELY. They can't sit in the laundry basket, they'll bother my allergies. They have to be washed and tucked away where the cats don't think they are just another bed to lay on. There is even one over my comforter since they lay there during the day.
I vacuum twice a week religiously. Wednesdays and Saturdays without fail. If my allergies kick up in between, I vacuum a third time (this happens during shedding seasons fairly often).
I brush them once a week (on the bed, just before I change the blanket) then wipe them down with a damp cloth. I wipe them down a few times a week, the brushing is just done once a week.
The only cat allowed in the bedroom at night is MOwMOw and that is grudgingly. If he wasn't so desperate to sleep WITH me, he'd be locked out as well. I have an air purifier in the bedroom. When I get home from work I put it on and shut the door. It stays on and the door shut until the next morning when I wake up. Book and Neelix KNOW they aren't allowed in the bedroom once the door has been shut. They don't even ask anymore.
After all that, honestly it doesn't sound like your allergic to your cats but something else. You're fine at home but at work in the afternoon your have a reaction and when you're out.... that doesn't sound like the problem is your cats.
I take zyrtec, so all the above plus the zyrtec keep me JUST fine most of the year. Tree Pollen is a huge issue so during early spring there is just no escaping the misery...... aside from moving to a brown barren place and that's not happening.