I think it's partially a consequence of the size of the game. Even if you have a computer more than powerful enough to handle all the processing, you still need to load the data to be processed. So if something like a weather effect comes up, if the game didn't have it already loaded in RAM, it will have to pause until that happens. Seasons would probably be particularly prone to it because of the weather effects it uses, if they are for whatever reason not always on hand for the game to use.
If the game itself is consistently running slowly, like it's continually pausing or it's jittery, and not just occasional sub one second pauses, then it would be more likely that your computer is struggling to run the game itself. I don't think any of the expansions really push the requirements up though, so I would definately suspect memory loading first.
A potential solution without installing an SSD to run the game on would be to use a flash drive. If it's large enough, you could actually install the game right on the flash drive. To save a bit of space (or avoid re-installing), you could also move game assets (the files for objects and effects, whatever you would like to load faster) to the flash drive and use symlinks (created in command prompt. I can't really explain it, you would have to look for a guide) to point to the new location of those files (they are used to make a file that exists in one place look like it is really someplace else, allowing you to move files without the game needing to look for them in the new location). Flash drives aren't SSD fast, but I have found they are at the very least twice as fast as a typical hard drive. They also have better random access characteristics, since Hard Drives also need time to physically move the heads into position, so you can not only read faster, but stat to read faster.
