I too have the new Xbox Series X controller, playing on Windows 10 and the controller also disconnects.
It disconnects most often when I'm in the menus in the skill tree/abilites/map menu.
Very frustrating. Please fix this, it's the only game this happens.