All of the adults in my main family have befriended the hermit. She used to get culled, but now, he doesn't, so I should invite him to Newcrest. He's still friends with them, so that would be fun!
If the hermit is not there when you go (and one of your Sims already knows him), you may have to call him or her to the current lot (the hermit's house. In my game, the hermit hasn't been culled in a long time, so he has a job, and my Sims had to call him home from work to become good friends with him and get the fungal infusion fertilizer recipe from him.