...... really fucked me up

That looks brutal. I got completely wiped by Huns in my first game. =(

After that, and on higher difficulties, I found I had to play more defensively and create some hard borders to fall back to if necessary, between mountains, coast, and lakes, since horde units can't scale walls.

Building garrisons behind rivers was even better, since basic archers firing into rivers from fortifications get +9 strength (+4 more if on high ground, for an archer strength of 25 to 29, or max 30 to 33 with veteran ranks). My 4 archer garrisons were able to sit safe in their castles and murder any hordes that wanted to choke my rivers with their dead. Also got me some great fame from deleting their military units.