I think infantry might be a little bugged right now. Playing as romans I could ignore siege weapons and just attack the AI through the walls making it super easy to win. In addition it looks kind of ridiculous having your troops wallhack the enemy and ignore their giant stone wall. Is this a bug in the game or am I missing something ?

Fortification adds strength to the defenders until you are inside the walls themselves. If you swarm them and they have only garrison troops walls won't do much other than defend against archers. So depending on the defense army walls + what kind of army you bring as offense can be a nightmare or not a huge problem.