So on the long term it would do damage to the neighborhood if I just keep stealing their stuff.
I worry about this too. Kleptomaniacs are fun to play but it's not really a good way to make a living.
The first time I played one I gave him a job as a criminal and sent him to the gym at night to get some skills. He looked around and saw all this nice stuff and nobody around so he swiped the big TV and both treadmills, nice cash.
Then the next day when he went back there were two other sims on the remaining machines and the stereo was busted and he didn't know how to fix it. The gym was kind of useless after that.
So the lesson is, don't steal from public places, you're only hurting yourself!
If you absolutely positively have to steal something, steal from the rich.
In the beginning of the game it's a really good cash boost though, just steal all the fancy cars. They sell for a lot and it's not hurting the game, the rich farts can just take a cab like everyone else.
Later in the game it's mostly for the entertainment for me, I still think it's funny when my sim visits a house at night, if I don't keep him busy all the time he might swipe something as soon as he's alone in the room, usually the drapes.