Sorry about that. This is a bug but will take a client patch to address; it will be fixed in a future build.
Mouse+keyboard users can work around it by left-clicking on the X and then right-clicking twice on something else, but I'm not aware of a similar workaround for controller users.
Cheers, Michael Noland