There are a few more aspiration milestones that are tracked even when switched away but I don't have a comprehensive list. The Body Builder aspiration will keep track and certain milestones in the Family aspirations will that I can think of. Some milestones in the knowledge aspiration will too...
This is actually a really good question and is probably something we could get the information together on per aspiration and per milestone. Could be really useful for challenges etc.