As the question states, I once tried doing this. There was a region near my spawning point that I wanted to put an outpost on, and they beat me to it, so I decided to attack. I smashed their tribes one by one for 4 turns straight, then I ransacked the outpost, and then... I saw a stack of 4 more tribes, and one more tribe roaming the land. At that point no AI has transitioned into Ancient Era, so I was wandering -- does the AI get infinite tribes and can it be killed during the Prehistoric Era at all? Or is it some kind of bug?

I'm on Metropolis difficulty, if that's relevant, btw.

IIRC, you get a new tribe if you lose the last one in the neolithic era.

I think it's correct. Because of that the A.I. can't be defeated during the neolithic. But it is the same for the players :)

As the others have said, it isn't possible to completely defeat them.  But you can force them into setting up an outpost in a specific territory (poor yields and few resources, surrounded by your own territories, etc.).  Or you can just dramatically set them back so they need significant help from competitive spirit era stars to do anything.  I tend to try to keep a few weaker opponents around with most victory conditions and one that has come out of the neolithic era behind everyone else is a good choice for that.

It's also possible to defeat them (with a stack of runners) right after they enter Ancient Era. A free second city early is very useful. They usually have another runner around somewhere to found a new city far away from you.

"does the AI get infinite tribes and can it be killed during the Prehistoric Era at all? "

This is the largest "bug" I'm currently "exploiting", you have a few options depending on the size of the map and all that but on a normal 3 continent with a new world map you'll have about 4 A.I. on your continent and the more "natives" you can control from the prehistoric era the better. 

Declare war on them throughout the entire game repeatedly never allowing them to settle a city. Can easily get around 1k gold per turn this way, per tribe.