If you were in a party at the time, you can't change the setting. Leave your party, and you can change the setting.
edit: On PC - Here's
edit2: The tooltip on Xbox advises you to change your Xbox Account settings -> IMAGE
Settings -> Account -> Privacy & Online Safety -> Xbox Privacy -> View Details & Customise -> Communication & Multiplayer