@willbefast: This mainly depends on how you stage entries of your state machines ?

Their 2 main advantage is:

1 - their comportment coherence. If you does something to the AI, if your states / transistions are coherent, then the AI is coherent. You shouldn't have much "my BFF AI just betrayed me because I stole one food from them".

2 - design ease & maintenance. If the BFF exemple happens, this should be fixed really easy if your state machines are documented (now, I know documentation in informatics is somehow "yeah, I should do this when I have time")

An assembly of state machines can produce the entries of another one. I remember doing this in my studies 15 years ago (this was electronics & telecoms. State machines are part of a chip/cpu design, using dirty conception langage such as VHDL, to study and design, say, a 200 entry state machines. And then, how a chip interacts with other environnemental chips.).

Staging entries helps to "normalize" states entries.

ie:

State machine A says when number X is over 9000 it goes to state A1. Say A1 says "I want a new system".

State machine B says when number Y is equal to 42% of something, it goes into state B1. Say B1 says "have many military".

AI main state machine M says when state machine A is in A1 and state machine is in B1 then it leads to M1. "I'm gonna be medieval on someone". And that "someone" is the result of diplomatic state machine D.

This way, if you want to create say Pearls for ES2, "all you have to do" (yeah, "all you have to do = lol") is creating a new state machine P for pearls, and review the main machine M, duplicating states, to include P comportements. This, without having to mess around with machine A and machine B.

The only problems of states machines is the non-unicity of actors leads to CPU explosion (you would have 1-7 state machines, if 1 machines per AI player -> so, I guess, 1 to 49 passes ?).

The benefits of staging here is machine A, B and P (if you just use ONLY ressources for thoses), have not to go through 49 passes, their result alway stay the same.

If you've got a diplomatic state machine D (aka the one that looks the D and M of the others players in the past) and the AI's own M machine may have to go through 49 pass. To define an intention then works with the intentions of the others.

This separates AI from being a traffic light automat to something that "thinks" somehow.

So for this AI design, staging can help to reduce a little some CPU usage there. AI have possibilities opened from their ressources, this have to be computed only one time. But diplomatics / political / whatever have to be computed depending of what an AI may know or not know about the declaration of the other factions.

You can reduce 49 pass to 14 this way.

1st pass to define an M => 7 pass

2nd pass to work with all M of other machines, maybe in a "futur machine F". A % of other M entries could be randomized because "SAWY, I didn't know you wanted to make peace ?", and a % of others M states could be exact, depending of factors you define in the game (empires are far but I'm probing / spying you and got ships in your empire but there was a long time no see, etc). This would define a new F states. So 7 new passes.

Then, if you want to play "I know that you know that I know" games, make another pass for this, using all F states. This time, "intentions of the others" and their possible reactions to yours can be studied. Here your empire would watch the press and TV of the other empire, etc. This would create a R machine, R for "real" and what your AI will really do.

Randomizing stuff helps to emulate stupidity, which is somehow non-easy with state machines. The base % of what the F and R machines know of others can be a slider left for what players likes in your game options. Say... 61% for a start :p

But maybe this is allready somehow how you do stuff and I'm just talking like this for nothing (and / or maybe I'm super wrong on states machines today since I just use those for C# webpages now) :)

mixerria wrote:
Have you ever played Civilization: Beyond Earth with the Rising Tide DLC? That is a kind of diplomatic system that works and feels very great as a player. Each action works towards a separate Respect/Fear-meter for each Faction. http://civilization.wikia.com/wiki/Diplomacy_(Rising_Tide) If you haven't heard of this, give it a shot. :) You, of course, can't copy that system, but a lot of things there are just too good to completely ignore them. Also, they introduced a Diplomatic Resource with the DLC which is very similar to Influence in EL. (Firaxis may have been inspired by you! xD)

Stalker0 wrote:
Yep. Again, it always about need. Do you want to make an enemy stronger? Of course not! But....what if it makes you stronger too? Then you would definitely consider it. Remember, if you and an enemy become stronger, than that means all of your other enemies are now weaker, which is also useful in global politics.

And that's why I don't like "likeability" to be a major factor in diplomacy systems....that's not realistic. You will deal with the slimy merchant if it gets you what you want.

Probably a little off topic but, for the record, we actually used multiple gauges in Endless Legend to determine AI behaviour: fear, trust, envy and war-fatigue to be precise. The issue was always feedback: if trust and fear  are both around 90, should I see "terrified" or "blood brothers" in the diplomacy screen? Whatever choice you make the AI will be unlikely to behave in accordance with what is displayed: it'll still act terrified while displaying "blood brothers" and still act amicably while displaying "terrified". This meant a lot of behaviour that only made sense to those of us who were able to peek under the hood

For ES2 we've opted for a system where the AI's attitude is a state which player actions can potentially change. The state defines behaviour and the state is what is shown to players in the diplomacy screen: what you see is what you get.

So yeah, there is no "likeability" metric that increases until the AI becomes your BFF: there are only states and conditions which need to be met for the AI to transition on to the next one

Kweel_Nakashyn wrote:

If AI's stance is too be considered as a "State Machine" (see what I have done :p), there is method to handle dozen of entries like this so your AI is not taking random decisions.

wilbefast wrote:

For ES2 we've opted for a system where the AI's attitude is a state which player actions can potentially change. The state defines behaviour and the state is what is shown to players in the diplomacy screen: what you see is what you get.

\o\ /o/ \o\ /o/ \o\ /o/ \o\ /o/ \o\ /o/

I hope there's other entries than player action that can change their state, because this is player-centric...

Kweel_Nakashyn wrote:

Kweel_Nakashyn wrote:

If AI's stance is too be considered as a "State Machine" (see what I have done :p), there is method to handle dozen of entries like this so your AI is not taking random decisions.

wilbefast wrote:

For ES2 we've opted for a system where the AI's attitude is a state which player actions can potentially change. The state defines behaviour and the state is what is shown to players in the diplomacy screen: what you see is what you get.

\o\ /o/ \o\ /o/ \o\ /o/ \o\ /o/ \o\ /o/

I hope there's other entries than player action that can change their state, because this is player-centric...

Bear in mind that state machines have weaknesses as well as strengths. In a nutshell it's going to be harder for us to take into account new player behaviours as we move forward

I'll give you an example from Endless Legend. When we introduced the Pearls of Auriga (resources that appear on the map and can be collected by armies) we needed the AI to react to players grabbing them from its territory. So we added a new "attitude modifier" which reduced the AI's trust in the player following an pearl theft and that was that. Attitude was a sort of stew-pot and behaviour was the flavour of the stew: we could take into account new player actions by simply lobbing new ingredients into the pot.

The advantage of ES2's more structured approach to attitude is that we won't end up with a flavourless mess when ingredients are mixed randomly together. However when the time comes to add a new attitude modifier (or "relation log" as we now call them) it will be necessary to consider its specific impact on each state and transition, rather than its impact on a set of gauges (eg. fear, envy, trust and war-fatigue as in EL). We're going to have to considerably more careful about the player behaviours that we log

Kweel_Nakashyn wrote:

AI main state machine M says when state machine A is in A1 and state machine is in B1 then it leads to M1. "I'm gonna be medieval on someone". And that "someone" is the result of diplomatic state machine D.

This way, if you want to create say Pearls for ES2, "all you have to do" (yeah, "all you have to do = lol") is creating a new state machine P for pearls, and review the main machine M, duplicating states, to include P comportements. This, without having to mess around with machine A and machine B.

That's an interesting way of looking at the problem. We do use hierarchical states to reduce the number of transitions necessary, but our architecture doesn't use this staging as you're describing it. I'll see if I can't read up on the subject a little...

The only problems of states machines is the non-unicity of actors leads to CPU explosion (you would have 1-7 state machines, if 1 machines per AI player -> so, I guess, 1 to 49 passes ?).

The benefits of staging here is machine A, B and P (if you just use ONLY ressources for thoses), have not to go through 49 passes, their result alway stay the same.

If you've got a diplomatic state machine D (aka the one that looks the M of the others actors) and the AI's own M machine may have to go through 49 pass. To define an intention then works with the intention of the others.

Reflectivity separates AI from being a traffic light automat to something that "thinks" somehow.

So staging can help to reduce a little some CPU usage there. AI have possibilities opened from their ressources, this have to be computed only one time. But diplomatics / political / whatever have to be computed depending of what an AI may know or not know about the declaration of the other factions.

You can reduce 49 pass to 14 this way.

1st pass to define an M => 7 pass

2nd pass to work with all M of other machines. A % of entries could be randomized because "I didn't know you wanted to make peace ?", a % of others M states could be exact, depending of factors you define in the game (empires are far, I'm probing / spying you, got ships in your empire, etc). This would define a new M states. So 7 new passes.

Then, if you want to play "I know that you know that I know" games, make another pass for this, with updated states from the last 7 passes. This time, "intentions of the others" and their possible reactions to yours can be studied. Randomizing stuff helps to emulate stupidity, which is somehow non-easy with state machines. And base % can be a slider left for what players likes in your game option.

But maybe this is allready somehow how you do stuff and I'm just talking like this for nothing (and / or maybe I'm super wrong on states machines today since I just use those for C# webpages now) :)

Appreciate the time you took to lay this out  As I'm a designer and not a programmer I'm going to have trouble going into too much detail with my response I'm afraid. I would say though that the difference between 49 and 14 passes is trivial if each pass is O(1)  By the time the attitude does its refresh the AI will already have crunched and cached a large amount of analysis data that it can access quickly and easily to make decisions.

You're probably aware we've been having some performance issues lately, but these problem aren't related to the exciting, clever stuff: they're mainly caused by the memory allocations of system calls related to data synchronisation between the AI and the game. Boring right? I'm afraid it's unlikely that the state machines are going to be the weakest link in the performance chain any time soon

I see what you're saying by O(1), I'm imagining this to be non-blocking calls. It seems to me someone made the same studies as me there :)

Non-boring :)

As much as I can "feel the game", without at all knowing the real problems, i'd say the lag is not coming 100% from what you say.

I'd say if there's a drawing problem from AI => game, the main problem for me I see is that it may be a combo of two problems ?

Problem A - the galaxy screen is lazy drawing stuff it doesn't see in fog, or hidden behind a 2D screen.

I have (almost) no lag when pushing the end of turn button when I push the button in a 2D screen, and the lag is super long when I'm watching things in the galaxy screen.

I'm not talking about -5% lag, I'm talking about -80% less lag. A big part of this lag comes from the galaxy drawing stuff.

I see this because when pushing in 2d screens then unveiling some "fog of wared" AI ships, they tend to move in my turn. They have not been drawn.

So the galaxy did not draw everything for a reason (and to me, that reason is bad).

I'm not sure to never have see this in EL, btw.

Problem B - as you just told, there may be a problem of mem sync.

I'd say the galaxy may ask full mem buses with higher priority than AI is writing its stuff. And that might be because the game calls the kernel drivers and the AI don't. So the OS would say "ok guys, just the kernel before anyone".

The AI would never signal "ok, I'm done, go on game, you can draw now". If you multithread AI and the game, this may be a super classic threading bug. Galaxy should wait for AI's signal to draw. And the AI signal would only happen once the states are stable.

The thing that makes me think this is when you reload. When you reload, AI states are wrote, and read as avalaible data. Being data, AI result is super stable. Then here, galaxy can draw very well from this with no lag.

Not "almost no lag", just no lag.

Again, my posts may be genius or pure stupidity, TINA :)

wilbefast wrote:

Kweel_Nakashyn wrote:

As much as I can "feel the game", without at all knowing the real problems, i'd say the lag is not coming 100% from what you say.

I'd say if there's a drawing problem from AI => game, the main problem for me I see is that it may be a combo of two problems ?

Problem A - the galaxy screen is lazy drawing stuff it doesn't see in fog, or hidden behind a 2D screen.

I have (almost) no lag when pushing the end of turn button when I push the button in a 2D screen, and the lag is super long when I'm watching things in the galaxy screen.

I'm not talking about -5% lag, I'm talking about -80% less lag. A big part of this lag comes from the galaxy drawing stuff.

I see this because when pushing in 2d screens then unveiling some "fog of wared" AI ships, they tend to move in my turn. They have not been drawn.

So the galaxy did not draw everything for a reason (and to me, that reason is bad).

I'm not sure to never have see this in EL, btw.

Interesting... We do like empirical evidence

Our profiling tools seem to point though to the synchronisation between the game and the AI as the source of the performance issues. This communication is decoupled from the user interface, so the screen you're currently looking at shouldn't have an impact. There may, of course, be a second performance issue unrelated to the AI...

You're right. I replayed a little since I posted that and the differences I noticed post EA (update "0") are not there anymore.

A little background from me:

IRL I'm a C# dev, coming from 15 years ago electronics & telecom studies. I know my studies are outdated somehow, for instance I used to learn doing my electronic stuff, then program low level asm drivers up to OS kernel, and not much kernel details. The only kernel details I know a little of is from NT and NT-RT edition, to bypass auth to test my stuff. And then making power shutdowns doing so.

(also, accidentally the banana plug into the 220V plug, power shutingdown the school & their like 10M€ laser microscope and their like super secret R&D white rooms, but that's another story involving a couple of banana plugs, air ohmic resistance, a rugby friend and a stupid blaze).

So I'm not "a little outdated", I'm like super outdated. And dumb sometimes.

This curses me with bottom-up thinking whereas modern dev thinks top-down.
In other terms, I'm a slow dev. Plus, I'm oldschool.

My CPU here is a 1st gen i7 920 from 7-8 years ago (bought it for Empire Total War : 2009).

OS is on a SSD.

Exchange file is on a physical Raptor HDD.

And the game is installed on a different Raptor HDD.

+ECC memory if I remember well.

This is an old electronic trick to optimize mem buses on a PC.

When the computer need the OS, it uses the mobo -> SSD bus, when it needs the exchange files, it uses the mobo -> HDD1 bus, and when it's a game, it uses the mobo -> HDD2 bus.

Since all buses are different, the mobo can use the 3 without waiting to feed & free a common bus.

Now, since i7 are heavily multicore / multithread CPU, data can be processed independently like that. Each buses is freed independantly, my bottleneck here is more CPU than bus (and for now, it's fine).

This typically gives 5-10% more perf on games.

I did not monitored it but the End of turns real hangs are like a few seconds, even after turn 80. And the total end of turn phase is like 15 seconds tops, +/- 7 seconds. 

I'd say a small half of the end of turn is a real hang, and the other big half, I'm free to navigate screens waiting for the rest of end of turn.

I did not test your last patch tonight, yet (I will and edit here to say).

Ok, so I passed a few turns @turn 95 from an update 1 > quick patch 1 > tonight quickpatch 2 game.

This was +/-17 seconds 4 times if I'm looking in my empire capitals (hangs in "pending" 10 seconds, free to navigate 7 seconds, then the game hand back the end of turn button to me) and 10 seconds 2 times if I'm looking at a lonely ship ("pending" 10 second hang and somehow 0-1? seconds to give the end of turn back).

I have a couple of strange bugs in this game, like a third hisho spawn so I'm not sure to have to start a new game or not since update 1. Maybe since hisho should spawn first for UE, anytime there's a new patch then there's a new hisho spawn. Dunno. Got also staying AI probes and my very first invisible ship battle so I should start a new game I guess.

Problem B - as you just told, there may be a problem of mem sync.

I'd say the galaxy may ask full mem buses with higher priority than AI is writing its stuff. And that might be because the game calls the kernel drivers and the AI don't. So the OS would say "ok guys, just the kernel before anyone".

The AI would never signal "ok, I'm done, go on game, you can draw now". If you multithread AI and the game, this may be a super classic threading bug. Galaxy should wait for AI's signal to draw. And the AI signal would only happen once the states are stable.

Hmm...

To be honest I'm not sure who has priority when it comes to memory access - the programmers will be able to say more. As for having the galaxy wait for the AI's signal: I feel it'd be a shame end up synchronising the rendering with the AI, as one of the advantages of multi-threading the AI in the first place is that it lets you carry on rendering while the AI executes. Or did I misunderstand your proposition?

Notice first that I'm super unsure of this : TL:DR : I'm unsure and hope I don't make you loose too much time.

This is just perception & oldschool guessing.

Your AI process is typically computing stuff, reading data coming from an exchange file and writing data to a second, or the same exchange file. This thread involves no kernel, it asks no OS drivers other than the mobo & cpu & mem buses & ram or HDD depending where you read & write data. Here, it's basic operations.

The game & the drawing engine, on the other hand, asks maybe all parts of the computer's electronics. So it is in deep needs of many kernel accesses and for this have priority over any other tasks the OS have to do. Maybe the OS reserve it 99.99% of mem buses for instance. I don't know.

The game says to the OS : "hey i'm the boss now, just like I'm the GPU boss, I'm the sound boss, I'm the mobo... Ok, not the mobo but almost the mobo boss.

So the game asks to write / unload textures to the GPU. Note: I'm not a big specialist there.

So I'm imagining things here :

The game says "I'm the important process, master of galaxy, hey mem bus, do not betray me and forget everything about anyone else. But still as the message #1 to AI, give me what AI said".

Other I/O : "hey"

AI says "ok, so"

Other I/O : "hey"

Game: "msg#2: Just ! Now ! I want it now !"

Other I/O : "hey, click, wut ?"

AI: ", in fact, according to my calcu..."

Other I/O : "wut hey, space bar, wut ?"

GPU: "TEEEEEEEEEEEXTUUUU..."

Other I/O : "lowut, hey, right click, wut ?"

GPU: "..RRRRRRRRRRREE"

Other I/O : "hey"

Game (prio): "msg#3: AI GIMME NOW WHAT YOU"

Other I/O : "hey"

GPU:"TEEEEEEEXXTU..."

Other I/O : "hey, screen stuff lolwut ?"

GPU: "...RRRRRRRRRRREEEE"

Other I/O : "hey SOUNDS, LOL !"

Game (prio): " SAID !".

Other I/O : "hey"

AI : "...lation... Err ? What this is not msg#1 or 2 anymore ?" and then AI timeouts.

Textures would get the bus priority because kernel say that GPU have better access to the bus than the game, or anything else big data a game engine needs. I'd say "texture" for this big data, i'm not familiar with game engines. So this may be gibberish. Also, when I say "draw", I mean it in a webdev point of vue : data and logic is there, but not yet refreshed. In the game, you need to draw AND animate. I just tell "draw", so don't take this only with the GPU process in mind.

The game mem stuff may also slow down the bus enough so the AI timesout.

Then AI thread times out, and the game would ask again then AI restart but can't speak, like, forever. Because the games load "textures/big data" that floads the bus. The important thing is by the time the ressource have more prio than the AI aka everything (GPU / Game / Screen / Sound / Keyboard & mouse), then your AI will always be the last.

It's just a small thread using CPU / RAM like anyone does : OS typically says "are you at least using mobo interrupts ? No ? So you're the weak one, just stfu". (I had this bug on some low level drivers in my studies, I had to waste CPU calling a mobo interruption and this was gone).

It would be interesting to watch CPU times with only the AI running + a dummy game engine that clicks 20 End of turn. And to compare to the CPU times of the AI thread running with the real game clicking 20 end of turn. Not ellapsed times, just CPU time.

It would also be interesting to count the number of each thread timesout in both situations because there should not be much differences.

For the sync question you say, I don't really understand how the game is suppose to draw AI results that are not there yet.

So, to me, the AI must absolutly finish its job before the game can even start to draw. By not coordinating the two, if your AI is not quick enough, and it may not be quick enough when the game advances say 100 turns and have a shitload of information to compute.
You would end having the sort of game - AI discussion I just told. The AI have a lot to process, and to tell. But the game would say "not quick enough" so either kills the AI, or the AI times out, or is deaf to the AI. Or whatever how the AI dies.

Or, I guess the game ignore stuff when it can't say what AI had to tell :

What I observes from the irl game is fog problems mainly. Ships acting strange when entering or going out fog of war. Or if you just zoom or click next to them : they just move. In my turn.

When I unveil fog of war on a ship, then the game sort of remembers it did not draw what AI said, so "in my turn", the AI ships moves.

It's very strange. The game may have timeout to draw some objects because it had no results from AI.

I'm imagining the game flaging objects like "will draw this later because AI said nothing or giberrish". The game put a flag on the ship, to draw when the AI data will be ready.

And when the player watches closely those objects, the game says "hey I did not draw this woopsie", then the ships moves. They move slowly, like you were in the end of turn phase.

It's like half of the ships willing to pass in the system you're interacting with is moving when you move ships there / from there : very strange.

It's unconvenient when you plan to attack a fleet that just moves out (Cravers' quest #2 aka "kill this moving ship" is a pain because of this)

The thing that makes me think this is when you reload. When you reload, AI states are wrote, and read as avalaible data. Being data, AI result is super stable. Then here, galaxy can draw very well from this with no lag.

Not "almost no lag", just no lag.

Again, this is interesting empirical data. I reckon the issue may be one of memory saturation though: the game cleans up when it reloads, so has a lot more de-fragmented RAM to play with. That's just my guess though, and again I'm a designer not a programmer

I would not bet on this, but I'm not low level dev anymore.

To me, ram frag is just an annoyance for modern OS.

Access to RAM takes, if my memory serves, err 1+1+1 cycles, right ? If fraged, +2 cycles without ECC memory (1 cycle with ECC ithink, but can be something like +4 10% of the time), something like that ?

Say your state machine results is 1ko, even if it's fragged in 1000 (which is very very very very unlikelly, since the data is supposed to be fresh), this is +0.001 sec top, not "forever".

Reason for this "unlikely" : if you ask a fresh 1ko (or even 100ko) part of mem, then the OS/ram (can't remember who tells) should serve you a defraged part of mem.
That's if you asked it with an end.

Sure you can write to it in RW style with no end. There the OS gives you stuff to write on, then frag you after, because, hey, you're not supposed to write in RAM for eons without closing access in Random Access Memory. Especially if the fat big game thread ask "MOAR RAM. JUST. NAOW.".

So when the game, or something else, ask RAM, then it ask the next part of RAM it can RW on. The AI say "hey my stuff ! Oh whatever, I timed out anyway", whatever can happen there.

But I'm unsure you R or W ram non closed like that: I wouldn't do that.

I would do like "hey that's AI speaking, I need 100 ko, no more no less, that's my 100ko, nobody can write on it but me and the game when it asks for that file."

And then, in those 100ko, use 50 ko for thing i have to read and 50ko for things I want to write, both part in the same format ideally, and secure/hang the game when AI is cleaning 50ko output data there. And the game should secure/hang the AI when he wants to clean the input 50ko of AI, if that's how the thing works (I don't know here : maybe your AI handles both parts).

Maybe get 200ko and use one file then the other. But still here, you have to make sure one is free before using it.

You can't rely on the AI tiny thread authority comparing to the other kernel demanding threads. If your AI thread doesn't have access to CPU time in the first place, it would never have access to RAM before the other threads. And the other thread may stomp over your file and you may have to redo.

About frag ram problems, I'd say if you have frag RAM, you may have like 0.05 sec hangs with that. Tops. It's a problem if you set 0.005s timeout for the game thread, yes, there it can be an issue, but this is unlikely I think.

=> count games & AI timeouts, you will see if there's timeout problems :)

Notice that even it's not your threads set to time out, maybe what times out threads is just the ordinal clock of windows.

There, it would be a kernel priority problem. Modern kernels don't panic anymore, well, not that much, but they tend to ignore a lot of things.

If you did not coordinate your threads by program and let windows do coordination for you, it's the best way to 

A - having those problems again when porting the game to other OS

B- having strange unlogic coordination bugs

C- BSODs.

I hope think you kept a little control there with you programming a coordination stuff and not having faith in windows.

Because : Windows. Faith.
Just two different words that never should be in the same sentence, like, ever. :D

Again, my posts may be genius or pure stupidity, TINA :)

People in the same environment tend to think the same way - having other perspectives is a good way of avoiding this pitfall, hence G2G.

Notice, again, that I'm 100% unsure of what I wrote. I

Hi there,

I did try to read through your post and compare it to what we do, but actually, C# is way higher-level than that or at least we use it at much higher-level ^^.

Here is the basic architecture layout that we are using for our AI/Game separation. At least, I hope it will give you a basic answer about why we’re suffering from some lag from time to time (like every time the AI runs ^^).

First, you have to know that we are running on Mono. And not a recent mono. Unity forked Mono some years ago to have a stable version to work with and avoid themselves the pain of integrating new Mono features into their software every year.

But they are now starting to upgrade the “Unity-Mono” framework they use because they understand that it is starting to be too outdated.

So you can’t trust that it works like a good old C# 4.0 (or even 2.0 ^^). Some parts do, but some don't. An example? Ok. Here you go: accessing an element at a defined index in a list will trigger a method call… which is not the case in C# normally. No big deal right? Sure, not until you’re accessing a list 10,000 times searching for properties in our simulation hierarchy ^^. This only difference can increase your search time 10- to 20-fold…

Back to AI: basically, what we decided some months ago (a little more than one year, actually) was to completely separate the AI thread and its working objects from the game. The goal was to give as much time as needed to the AI to choose an action without having the game lock up while it waits for the AI.

In order to do this, we have to guarantee that the objects used by the AI during its decision-making process do not change.

The simple way to do this is to create a copy of the game on the AI side. We call the AI’s copy a “snapshot.”

So what we are doing is:

1. At some point, we decided that we need to launch a new AI decision-making process.
2. The AI sends a message to the game saying, “I want to synchronize.”
3. The game acknowledges this and changes its internal state to “WaitingForAISync.”
   • In this state, we wait until all pending “orders” are executed. Orders exist for every action that the players can perform (Queue a building in a given system, move a fleet to a specific node, etc.).
   • Next we move to the game state “AISync” and we send a message to the AI saying “StartSync.”
   • During these states, the game is not allowed to process “orders.” So when you, as a player, click on a button, the order is put in a queue to be executed later. In theory this should just mean a short delay between your clicking on a button and something actually happening.
   • Meanwhile the game will continue to render the GUI and 3D with the same priority as normal, so you, as a player, should not experience a freeze.
4. The AI asks the game for the list of all the living game entities (ships, systems, galaxy nodes, pending research, available research, etc.)
5. For each game entity, we send a message to synchronize it:
   • We store, on the game side, a “contract” for each entity type which explains how to access the data from the game entity object.
   • We store, on the AI side, a “contract” for each entity type which explains how to create and update the “snapshot” of the game entity object.
   • Once both contracts are exchanged a memory stream can be used to send the data across: the game writes to it and AI reads from it.
6. Once all the game entities are synchronized we start the AI decision-making process and we send a message to the game saying “Sync ended.”
7. The game changes its internal state to “Main” which is the default state when you are not finalizing the turn or anything like that. The game now starts to execute orders again. On the AI side all the necessary computations are performed using the snapshot and, when we think we have a good “decision”, the AI sends and order to the game exactly the way a player does.

A little diagram could help I guess:

This architecture allows us to separate the AI’s computation from the game logic. As a result, the AI can take longer to execute than the duration of a single frame without causing lag and without our needing to make it interruptible (so that the AI can be executed and resumed over multiple frames). So the game can thus run smoothly while we are performing big computations (not that big actually, but still…)

The only bottleneck between the game and the AI is the sync process. It can take some time to go through all the game objects, to get the data, to write it to the stream and to wait for the data to be read into the AI’s snapshots.

We are using threads and trying to reduce the number of objects synchronized as much as possible. For the moment, we’re experiencing something like 0.2 to 0.6 seconds of synchronization. Which is both huge and not so huge ^^.

Once the AI has been synchronized with the game everything works rather well, because our entire game architecture is built to handle asynchronous orders for multiplayer. To the game the AI is just another player throwing orders at the server every now and then.

So, what doesn’t work? Well…

First, we’re having trouble waking up the AI at the right time. So when you start a battle with the AI it takes some time before the Sync starts, then more time for the decision-making processes to execute, then some more time to transform the decision into orders, then more time to send them back to the server… In the end, if you start the battle right after a previous AISync, you could wait 1 or 2 seconds before the AI answers the battle… And that’s only if we’re able to give a satisfying answer on the first try (in the case of battles this should always be the case, but in other area the game-state may have changed by the time we reply ^^).

Second, we’re have some problems reducing the duration of the synchronization. Really it’s just a matter of taking the time to optimize the process, but with all the features and tweaking and stuff, we did not have much time this at the moment. We hope to do a first pass at the beginning of next year.

And last but not least, we are experiencing lag since the last update. We have more or less found out that it is due to the Garbage Collector triggering and blocking everything for about 0.2s. Which is huge.

As the garbage collector freezes all the threads, even the main thread (where the game is running/being rendered) the player can feel the freeze. This process triggers without warning and without any way to ask it to take less time per run...

We have discovered that one loop of AI synchronization and decision allocates on the heap more than 20 MB in less than a second. And these 20 MB are always freed after the decision, meaning they’ll need to be collected by the GC. Which is huge, and is more than a surprise to us…

So we have made a lot of memory optimizations (pool of messages, use of structs instead of classes for temporary information, etc.) in the last few weeks to reduce the “temporary” memory allocated on the heap. But we’re still struggling to find THE big mistake we have made (because, 20 MB has to be a big mistake, doesn’t it?).

We found last week that something between 2 to 4 MB are allocated during the Synchronization (reading data from game entities to write them to the stream and reading from the stream to write to the AI snapshots). So that is some memory we have to free but it’s not THE big allocation… In a way, it’s kind of reassuring that the way we exchange data between the game and the AI is not the main problem. But that doesn’t help us find where the leak is hidden ^^.

Again we are planning on throwing all we have at the problem at the beginning of next year.

So, to summarize:

• The lag we are experiencing since Update 1 is due to memory garbage collection. At least that’s what we have measured.
• The fact that the AI use more than 20 MB per synchronization is one of the factors that triggers this garbage collection.
• So our next job is to reduce the AI’s heap memory allocations. The goal is to reduce the need for the garbage collector to trigger.
• We also plan to work with Unity to understand why it takes that much time to garbage our memory and where it’s all coming from.

At the same time, we are working hard to keep the AI up to date with the new game features. So you guys can still enjoy a game against the AI when the update 2 comes out!

And let me tell you that’s a challenge, there are so many features! ^^

Cheers!

Flo (And Frog' and Will' as reviewers! Thanks guys!)

@ThorTillas Many many thanks for the explanation. You are clear as crystal.

I did not know about Mono framework so I was imagining low level pgming stuff.

About your architecture, it's nice.

A] Take the following with a big pinch a salt : if you have trouble to wake AI, to me, it's because AI is a CPU/RAM/mem buses thread. It have low prio compared to the game asking a lot of time very very high prio kernels drivers.

It may sound stupid, but I'd try to read the input of mouse (for nothing) in AI here and see if it's still sleeping. Mouse use the interupts from the mobo, anything listening to it will subscribe somewhere in the OS I forgot and have higher prio than other threads (or maybe I forgot too much :) ).

Beware, this would NOT be THE solution. Each interupt from the mobo would say "hey hey, right click lolwut?" to 2 threads instead of one : the game would be choppy as a result but at least that thread shouldn't sleep anymore. I think this phenomenon was studied by better dev than me so maybe you should look there ?

(sadly none of my colleague could not confirm this when I asked the question: we're in e-shopping middlewares, so no RT stuff for us).

B] What I would have done nevertheless is (maybe) add a data storage layer between the AI & the game.

Not for asynchronism, for memory allocation issues.

It seems to me, and it may be a telecom reflex from me, that most quick/slow/quick/quick/slow erratic and random behaviours when passing information from one side to side have better behaviour with buffers.

I will ask confirmation for the following with my colleague:

For me, if messages between threads are by RAM, you may control the exact moment you ask something to be written in RAM.

Usually, you tell the runtime "tell this in RAM", and the runtime say "ok, done", when it is actually meaning "ok, I'll do that". You're not controling when that stuff is actually written in RAM. And also you don't control when you read that stuff you're waiting for and also, but with less problems here, the exact moment when you actually write it on the stack.

Sometimes (err... most times) things will be done when you want to read it. The runtime would say "oh yes, I had homework err just wait for me, yeah err.......... done".

You must expect lag effects like a train accelerating or slowing down : first the locomotive accel or slow down, then 1st wagon, then next etc. It's not the whole train at once.

Here, buffers solves many chop issues. Chop & sync problems like the ones you have reading that youtube video with a 0 sized buffer.

This have been studied again a lot I guess ?

C] 20 Mb may be the size of your contract objects, did you try to dispose them at the end of the syncing ?

(not finalize them but dispose them: if I'm not wrong dispose are hard calls to the garbage collector. destroyers & finalize are soft calls so the runtime say again "yeah, yeah, I'll do that" and actually make this well, at random times. When you ask some free ram for exemple.)

(because doing 0.2-0.6 (or even 1s) disposable work when the game orders engine is freezed is ok, no ? Or you need contracts until the end of decisions ?)

Ok about those 20mb, the last one may be too easy & you should have allready checked this a million time:

Maybe it's the size of a snapshot, maybe there is a copy problem there ?

(like :

Snapshot newssnap = new Snapshot(oldsnap) ;

newsnap.update();

// ... do AI stuff reading and writing on newsnap, and maybe reading oldsnap

oldsnap = newsnap;

return; // there, both objects exists until you do this. Since newsnap have no meaning, GC would destroy this after an AI cycle

)

Are contracts using their own threads and if so, what are their mode ?

(Since I'm rusted on threading, I'm reading threading stuff and they say each user-level thread reserve 1Mb of memory. I'm reading user-level / kernel-level syncing, it's not really what is the easiest to read about, but it's really interesting. I'm imagining the game using kernel mode and the AI user mode but only you could say).

This is where Stellaris focused on adding end-game events, such as crazy zergs and extradimensional invasions, as well as Fallen Empires becoming active super-powers.

Since human optimization and heuristics will always beat AI (at least with current coding techniques), I see no other way to make you scratch your head in the end-game, except of shoving a huge fleet of new aliens down your doorstep.