I had something similar to this happen once before. Out of nowhere, the wife had a fight with her husband because he was cheating. I looked in his memories and he had a romantic interest in the past that wasn't her. It was before the wife, though. Even though he never interacted with her, I guess it still counted, since the romantic interest never really 'ended' (though she wasn't even in his relationship panel).