The answer is yes! You can take them as early as infant, but they just have to lay on the ground while your Sim is fishing. That's not really a problem, it just looks odd. At the moment, there aren't any portable infant devices in the game. It's better when they are toddlers because they can sit and play. It's good to have a teddy bear in the toddler's inventory so you can click on the bear and the toddler can play with it. It keeps the toddler occupied, plus increases their fun and social bars. If you want to bring the baby or toddler, have the parent pick him/her up then click to send the parent to the fishing spot. Sometimes it can be a little tricky because the parent will forget and put the baby back down and you'll get the babysitter message. But if you use pause or just be careful with the timing, it will be fine. Also, sometimes the destination has a "visit with" option and you can just select the baby there. It may not be quite this easy fishing with babies at the beach because there are so many spots and they aren't really actual public lots.
It's even easier with children and teenagers. Children can even fish with their parents and learn their fishing skill early. In these cases, you'll need to use the "visit with" option on the location. Or just send the kid there on their own. Teenagers, of course, are just like adults with fishing.
I hope this is helpful to you!
