I guess the matchmaker joins you mid round to balance the teams. Maybe it helps trying to see it from the other side: If the system would never fill mid round all games would end with uneven teams towards the end. Sure, that still happens sometimes, but not really that often considering that on average 20 people leave a round of CQ during its lifetime (I think that was the number I saw the other day).