I can't presume to really know your situation, but from what you have described, it certainly sounds like Viktor is happiest with you. Being picked on by another cat in their own home is extremely traumatic for an animal, imagine living your entire life not knowing if you were safe or going to be attacked if you lost vigilance for even a second.
It sounds like he doesn't spend much time with people or other cats at your parents home, so it doesn't sound like loneliness would be too much of an issue, especially when he seems to store up his affection for when you are home.
Ultimately it seems you love him very much, and that's what a kitty needs most. He sounds like he's happier in the small space you give him than he is even when having access to the outdoors (and really, the outdoors isn't a safe place for a kitty to be unattended), and a life of fear and frustration is I'm sure not something you want for your kitty. It sounds like he would enrich your life, too, so it sounds like a great match
If you're worried about taking the outdoors away from him perhaps you could take him for walks? I've had great success with getting my cats to walk on a leash. Getting another cat as a playmate is also an option, if you have the money for it (I find cats don't need *too* much space, and typically you're good if there's at least one room in your house per cat).
This is all my uneducated opinion, so all I can really say with confidence is that I wish you and Viktor the best of luck