You should definitely say something back. Something he hates someone else says about him. Let him feel the same way.
