Yes, I've seen gems around the high-$ residences. However, I've noticed there are better places to more consistently find more valuable gems. When I send a Sim out collecting, I only pick up yellow sapphire or higher grade gems. You can generally tell by looking at the color of the object on the ground - if it's light blue, red, green, or gray my Sim doesn't get a cue to pick it up. Try looking at Stonehenge (usually at least one good one), around the Science Facility, and in the hill's between the stadium and the beach.