We created a How It Works video on our Matchmaker. You can view it at the end of this message.
Random Battles and Co-Op are Random. It's first-come, first-served and the video describe how players to join the queue are sorted out to create matches quickly. Our Matchmaker isn't programmed to understand winning or losing so it's just placing people on Team A or Team B.
Ranked is also Random, but there are 3 "Leagues" that players can sort themselves into. The Matchmaker will only pull players from the League you are in, so if you'd like to sort yourself into a smaller league, Rank Out on the pass the Qualifiers to move to the next one.
Clan Battles places you against other Clans based on your rating. At most, you can be matched 2 Leagues away from yours, so a Hurricane Team that is in queue long enough could be matched with a Storm Team (Hurricane -> Typhoon -> Storm) or vice versa.
Brawls are a temporary mode which factor "# of Wins" into account when attempting to match players.