Perturbee, I hear you and I am also not very happy about how this was set up by the creators. I do not use Mod. Doing and maintaining a garden takes a lot of time. This hobby needs micro management and eases up somewhat when your Sims has acquired the "tend the garden" level. But because you need to click on every plant when you are fertilizing, I found myself pausing the game quite often. And if you are playing more than one Sims there is a challenge there. But I like a challenge and gardening is one of my favorite in Sims 4.
Depending on my game play, I usually transfer to the fridge or to another family member what I need or wish to transfer. Then I cash the rest if I have enough fishes to fertilize my plants. If not, I keep just enough of one kind of harvest (say: mushroom) in my inventory. That way it simplifies the scrolling down the inventory to find what you are looking for. What I still do not understand is after you clicked to fertilize a plant it shows say 1 mushroom followed by say 4 mushrooms. So be aware that if you click twice on say 1 mushroom, your Sims will question you (Hey man! (waving), what your doing? (Lift shoulder) and proceeds to do the next queued action.)
I hope this help you a bit.