So far I've seen more variation in the female townies than in the male townies, but the facial structures differ enough to keep things interesting. The problem that frustrates me is that most of the game-generated townies are in the adult life phase, not the young adult life phase, meaning they make unsuitable spouses for most challenges that involve reproduction (since they age up so quickly.
However, the "manage households" tool in combination with testingcheats allows you to enter CAS in fulledit mode so that you can edit the townies in your game if you don't like the features that they're giving you. I'm not 100% that this is legal in challenges, but check the individual challenge to find out if it's important. Appearance affects little, so I'd think that as long as you leave the personalities alone, you should be good to rock and roll!