I have read many people online in forums who would like to be able to remove the Settlement Decorations.
At the moment THERE'S NO WAY TO REMOVE SETTLEMENT DECORATIONS once set.
Just add a button in the decoration selection screen with the choice "Nothing" or "Empty".
Make some people happy in a future Title Update, thank you.