I've been a bit baffled at this as well, unless the townies are really just that suspicious of me.
My sim will be innocently talking to people, trying to make friends, and doing the usual Chat, Gossip, Get to Know, etc., when BAM! My Sim gets a Naughty reputation for not being "entirely faithful."
I would definitely understand flirting, but I'm doing the friendly and joking (not flirtatious joke) stuff, so why is that the other sims think that my Sim is being unfaithful??