Gotcha, thanks! If it's only happening when you're playing without headphones I'm wondering if it might be an issue with audio settings. It sounds like the chat audio might be going out on a channel that wouldn't be coming through your TV speakers.
Have you checked your console's audio settings to see if it's set up to match your speaker configuration? If it's putting out audio to channels that don't actually have any speakers to go through that might be why you can't hear certain things.