Hello @TunnelRadio, thank you for sharing your feedback with us!
If you dismount and not set your horse to stay, he’ll follow and try to engage in the fight in the same manner as a thrall would, which is intentional.
Being able to hurt your followers is also an intentional mechanic of combat, certain weapons and attack animations are more ideal than others when fighting together with companions.