So it's commonly assumed that all the characters who cameo from other shardworlds are worldhoppers, yes? as we don't know where this power comes from, or how alot of it works, not every character we see cameoing in other books (Demoux, Gallod, Mraize, etc, etc) should be worldhoppers.
What i think is actually happening is that only some of them are actual worldhoppers (hoid, nightblood and other important characters) but the others use either (A) a device that allows them to enter the cognitive or spiritual realm, as distance is different there, allowing them to go to different shardworlds (similar to how Jasnah's transportation surge works) or (B) that a worldhopper can take multiple people they are linked with (eg: touching them, or someone else who is touching them) and they only have a couple worldhoppers.
if it does happen to be option (A) , that would explain why and how the 17th Shard can get to other shardworlds easily. Additionally, i think that worldhopping is orginally a yolen-based magic, although we see non-yolen's using it.