There's no way for duos to match only duos at all, they need solos or trios to work. Same with trios. That makes the option to only ever allow 3/2 vs. 3/2, which isn't a popular enough combo unless we allow really wide skill gaps in the matches, or really long wait times, or fill up the cracks with solo players.
As far as unfairness, that depends on your definition of fairness. It's 100% possible to make a fair match that includes both solos and duos, or both solos and trios, in terms of both teams having a shot at winning.
If by fairness you mean that the solo players will have less cohesion than a duo or trio, I don't know if that's necessarily true. There has been some evidence that 2 solo players are more likely to coordinate with a trio than a duo would, because the duo is likely paying more attention to each other (being two friends in a duo) than they are to the trio. So I don't know that we can say for sure that having solos mixed in with parties is more or less fair, and it significantly improves match quality in terms of fairness and speed.