I've run into this twice now. I have perfect food stocking my frig and the food my sims cook is almost always perfect, but after I put away leftovers, the quality changes to normal and the next day, my food is spoiled. (Sorry if this is too wordy, but I want to cover every aspect in case there's one thing that is known to be a problem...)
In Dragon Valley, I have a 64x64 lot with the maximum number of sims, 2 horses, 1 dog, and 1 cat. I play "Lord of the Manor" with one large manor house for the lord and his lady and their child, a multi-level barn for my horses and their caretaker/warlock living upstairs, and a cottage in the back for one of the lord's knights and his wife and child. My gardener/fairy lives in a fairy castle placed near the garden. I have placed a refrigerator in each of the dwellings (3) plus a deep fryer in each dwelling (3) so the men can have their deep fried turkey legs for breakfast. :-) I have one oven in the manor house, but after the Indulgent Bakery came out, I placed one of these ovens in the knight's quarters. I have almost every plantable in my garden, so carry a large inventory of ingredients in the refrigerator. The warlock also prepares potions, so I have a hoard of ingredients and potions in their appropriate storage cabinets on the lot. Keeping up with all the spoiled food became frustrating so I stopped playing this game.
I also have a game I started in Hidden Springs specifically to play the Indulgent Bakery venue and have a large inventory of ingredients in my refrigerator, from the Grandpa's Grove set I use to run the farm. (You have to unless you want to pay for each recipe you make in your bakery.) There are the maximum number Sims in residence (including plumbots to keep everything going), 3 dogs and 1 cat. There were four refrigerators on this 64x64 lot, a deep fryer and ice cream maker, and my food wasn't spoiling overnight. I ran into trouble keeping the bakery I owned stocked, so decided to build an additional kitchen to do my baking at home so I could use replicators to speed up the restocking process (twallen's Cupcake mod wasn't out yet) and added the bakery ovens and refrigerator to my at-home bakery kitchen. Now my food spoils overnight here as well and I can't stop it, even after paring down the number of refrigerators and removing an extra oven I placed in the bakery kitchen.
Can I be overloading my lots with appliances? Or premium content items like the playpens and walkers? I have had hundreds of ingredients stored in my refrigerator in multiple games because I have always loved gardening and usually play with several different refrigerators on one lot for convenience, but haven't had my meals spoil overnight until now. I have always played large lots crammed with stuff. I also use mods and cheat enough money to buy high end refrigerators and ovens.
If it's ok to mention them by name, I use the Cooking and Ingredients mod and twallen's Master Controller and Story Progression mods. I've checked all of them to be sure they are updated, and they all state there are no conflicts with the bakery. I haven't used twallen's Cupcake mod yet.