In reality, people can make less good-looking trim combinations, but often they will make things that look good to them. And I think the great thing about this is that what you might expect to look "bad" can be transformed into something surprisingly good by players.
I have seen a few people criticize armor trims due to many combinations being ugly.
To me, this is very Minecrafty. You can build very ugly houses made out of dirt or diorite, but we don't prevent people from doing it.
Banner patterns are the same and better for it
I didn't think the Vex armor trim would look that great (to me) since it looks a bit silly. But some people have used it to make armor that looks like clothing, and that's amazing. You couldn't do that if we forced more specifically tailored visuals. We give players the power.
For the record, this feedback is great and I love seeing different perspectives. I just think that this aspect of the feature is part of what makes Minecraft different to other games, in a good way.