I'm not sure where you're located, but I'm lucky enough to have several large Asian grocery stores within driving distance. They have all sorts of interesting things in Asian grocery stores (it's where I regularly get fresh heart, tongue and gizzards; and they have lots of other interesting things like beef lung, pork uteri and intestines, chicken feet), so if you happen to have one nearby do check it out!
I hope you have luck at the butcher or one of the slaughter houses. Some people are lucky enough to get good deals buying the less desirable cuts of meat directly from local butchers. I haven't been so lucky but it never hurts to ask.
Gizzards and tongue are considered muscle meat by the way. Kidney and spleen are both organ meat, and I feed my cats both for their 5% other organ needs. They seem to like kidney more than spleen though, for some reason.