Yeah, you need to press and Hold. It’s the same for ‘Toggle Stats’ where you have to press and hold, otherwise it opens the hero select screen.
I also didn't find this obvious, so thought there were issues with overloading button mappings! It either needs a tutorial pop up to explain this. Or a clearer message like [Hold] written after the icon name
There are however, other bugs which are mapping issues - such as:
left click to sort your inventory resetting your stats( believe patch is live for PC but not Xbox yet)
pressing Y for tower info brings up the map.