Am I just unlucky or do mana idols not drop until later?

Unlike equipment affixes, idol affixes never have a minimum level to start dropping, only idol shapes do. This is to prevent it ever becoming the best strategy to farm idols at a lower level to remove the chance of rolling affixes you don't want that only start dropping at higher levels.

This isn't a concern for equipment, because there are many other aspects of equipment drops that get better with levels, such as average tier, average number of affixes, average forging potential, etc. So, it's ok for equipment affixes to have minimum drop levels.