I've had that problem. I thought it might be due to my computer being so weak in the area of video card; I have just the basic card.
I had to delete one family which was permanently stuck, which ticked me off. I hadn't saved them in any backup files and ended up having to delete the town.
I've had it in SV and in AP, but not as bad, just for a second a couple of times. In Riverview, it was constant with that one family.
I've just started to save often, with the backup in case I have to delete that play time.