I think some of these interactions can be chalked up to University if you have it. Non-played sims are constantly slammed which leads to some awkward encounters. This is obviously very annoying because why would want my entire town juiced all the time?
Witches annoy me, mostly because of how long it takes them to level up. Seriously, I just created a witch, set all her needs to static and high. It took her until Friday, FRIDAY, to max out her witch powers. She spent six days doing nothing but magic (high level magic too) and it's just, argh, I can max out a fairy's abilities in three to five days doing other things and a werewolf's in about week doing other things. Generic face townie soup/ broken SP is probably the most annoying. I would love, LOVE to see generations upon generations of DV kids running around without having to do much myself but no, that's not allowed. Sims not letting me cancel betrayed or break up or other such interactions are also annoying. I was playing them as an OT3 or treating one of them as a literal DNA donor - in my mind, their partner is fine with it. One last thing is that sometimes when townies get mad at one household member, they take it out on all the household members. Like, it wasn't this person who was rude/mean/woohooed with your wife, why are you mad at them? The mad sims sometimes won't even talk to their friends for weeks and if I go over to their house, I get the 'so and so is behaving inappropriately/is being rude.'