I assume less than 3% of the players know/remember to check the secondary mode before queueing
I do think this is a problem that has to be addressed by making the special mode more prominent and immediately visible on the main menu, rather than buried in a sub-menu. Sorry for tagging you u/trichoglossusbee but idk how else to pass on feedback like this.
Can I ask what region you're in? I'm in the UK so on the EU servers, and I can find a special mode game easily anytime from noon to midnight (don't know about morning), and it becomes tough to find games after midnight. Main show is pretty much fine any time of day,