This is more about the children than the cats. At the shelter where I volunteer, when families come to see cats and adopt, almost always the cats (even the really outgoing ones) will sometimes take a defensive stance with children. It's a great learning opportunity for kids and we spend a lot of time with the children who come in, because they really want to love the cats. Teach them to not make sudden disruptive movements. Don't reach out to pet the kitty without gently offering an open (limp fingered) hand first for kitty to smell. Just because there are toys around, doesn't mean the cat wants the ball thrown at it - sit down on the floor with a long fishpole type toy and move the toy slowly along the floor without waving it wildly. Even young toddlers can be taught "gentle" and "slowly".
Once a young child and cat bond, it's one of the most beautiful relationships there is. They will grow up together, being partners-in-crime, playmates, cuddlers, and companions. But there is usually some work involved at the start to get the cat to trust the child and the child to learn how to treat the cat.