"Back" Button or Key

I don't see a way to press something to go back. Not a level up, which the right click or esc does, but back, to the previous screen. 

Like when you're in the Empire Mgmt screen, dbl click on a system to check something and want to go back to Empire Mgmt screen. If you hit esc/right-click, you go back to the main screen... Or if you are looking at trade routes and click market then want to go back to trade routes without clicking the tab. 

Or you were doing something, then went to quickly check something else but got so "immersed" in the side thing that you forgot what you were doing before that :)

The "back" button on the mouse or "backspace" on the kbd could do I guess.