Armors are terrible

I pick a different set to play with each run through and wear only that set once I manage to find/craft it. If you can pull yourself away from the min/max meta there is a lot of fun to be had. If you feel the need to swap armor sets for every different activity just to shave just a bit more inefficiency from your character, it's no wonder you don't like the sets.

There is supposed to be a wardrobe function in the next major update but even if that means some sort of quick change feature, I will probably just stick with wearing one outfit per play-through. I definitely don't feel as though there is any one armor set that is the MUST WEAR outfit. If you wear the xp bonus set and I don't then the only difference is you reach endgame a few weeks sooner than I do. Meh. I'll play the extra few weeks.