Yeah man. I think the "jumping tracks" issue is slightly different, wherein the person interrupting is trying to change the conversation completely. That's also annoying as fuck (although if that's not what you mean, then by all means, please clarify).
I am referring to, let's say, a debate...