There are plenty of reasons that someone may need to drop out of the match for.
Granted, some do drop out because they are losing and that is absolutely poor sportsmanship and if it affects your experience enough you can go ahead and ignore/block them.
But then there are valid reasons too such as internet drop out, a platform level crash, dashing to the loo, a piano fell on the house, they just won the toilet roll lottery (okay, maybe not them last two, but still!)
And while it may be aggravating that it instantly puts things at a disadvantage for that single moment, there will always be another match in the grand scheme of things.