I have not seen anything in my gameplay time to suggest you lose damage from using a silencer. I want to say recoil feels better but I haven't sat down and tested it to see if it's that much of a difference.
on hostage rescue silencers are a necessity due to the fact any sound alerts the enemies to kill the hostages. the one thing I do see them help with is if you silently enter a room and take out the people in the room without them firing first, it won't alert the people in the next room over. That being said, the AI is very quick to shoot you. Only way silencers benefit in my opinion is if you're doing hostage rescue and are able to one tap EVERYONE you see until you reach the hostage room, then you can flash/stinger breach that room.