Sorry, kj, I totally phrased my previous post wrong. What I meant was:
If you uninstall the patterns from your game, of course they are not going to be on your previous houses that used to have them, and will therefore had to be replaced by the game with something else.
And like Vamp said, it has to be a fresh build so you don't have to uninstall if from your game, you just can't use them in your building for the challenge.