Hi,



I've been lurking on these forums for a few weeks while i consider the options for AI modding. Specifically this comment from the devs peaked my interest:



"AI can also be a huge aspect of modding. Right now, we are finishing implementing the last high level code for the AI. The architecture seems to be satisfying, but we have exposed tons of parameters to finely tune the AI during the upcoming months (and we will need your help). I believe these parameters could also be exposed to mod the AI behavior."



I am a PhD engineering student with an interest in optimization, I also work with a guy who is exceptionally good at it (far better then I). Optimization of one or two parameters is typically straight forward, mainly because as humans we can easily visualize a search space in 1-3 dimensions. As you increase the number of parameters you move the goalposts for successful optimization way out of reach of our puny human minds. What I am proposing is to take a more mathematical approach. The most effective approach to this problem in my mind would proceed as follows:

• Identify which parameters relate to AI behaviour
• Define an objective function, that is a function which returns a measure of "AI quality", this is the hardest part. What criteria makes a good AI? And how do you recover numerical values from played games to enter into this function?
• Create a Reduced order model defining the relationship between AI quality and AI behaviour parameters
• Use the Modified cuckoo search algorithm (http://code.google.com/p/modified-cs/) developed by my colleague to efficiently search the parameter space for optimum parameter combinations
  
  

This is not a small task, many (MANY! (MAAAAANNNNY!!!)) games will have to be executed to define a good reduced order model, in an ideal world I would like to have the AI play against other random AI using a game client with accelerated time and no graphics output, simply statistics output from the games conclusion. In addition to that I would also like to have the AI play against as many different human players as possible in a similar fashion. For this to work well it is crucial that a good AI quality criteria is defined, furthermore this criteria should be calculated based on statistics from played games, does the client already output such statistics? The result of all this can mean many things, the hardest AI possible, the most balanced AI possible, the cheekiest AI possible...



At this stage I am mainly seeking advice and opinions. If any devs/experienced modders would care to advise on how I can proceed with this it would be greatly appreciated.



Cheers,

Bruce

Well mr.Engineer let me tell you what. if you are so smart why you just dont do your homework and discover everything yourself. It took me only 20 minutes and i aint any phd at all. Ask my cow Marty.

Finding the parameters in the xml file obviously isnt the biggest challenge. But defining a criteria for good AI behaviour requires constructive discussion, my idea of suitable AI behavior may not necessarily agree with your ideas. There is also very little reference material available which is why i feel it is constructive to simply ask questions.



Please restrict your replies to constructive criticism only.

In future if you don't want others treat you like a dirt you don't start with: "I am damn good at this and my friend is even better."



Then again I am worst possible person to say that, I myself use to start posts with: "yeah, I know more about this" and end with: "if you can't make it happen you are awful person".



Anyway problem with AI modding is the way AI is done in this game, as both you and Psilon said AI in this game is all about changing couple parameters... and that kind of AI never can compete with human.



Sadly it's too hard to program AI which could learn from games and could in the end give competitive challenge to humans.

(then again seeing how we have PhD engineer and his magnificient friend here that would be good goal for them... after all, it doesn't need but bit coding







Btw Igncom1 (or was it Ibise) week or two ago already made AI tinkering for Terrans which did work nicely... it's shame that it has lost in forums.

Lol i didnt expect to offend people by mentioning my dayjob. I was simply trying to put my experience in this field into context. Obviously if my field of study was Theology I could forgive you for calling me a douchebag for bringing it up.



You are quite correct, in that it is difficult to program any type of artificial intelligence. What makes me think this is a good approach is that even with simple parameter based decision making you can gain some interesting emergent behavior. Example: When the decision was made not to allow AI to retrofit in previous patches (obviously now fixed) I wander if anyone expected the AI to keep their redundant ships for fleet spam instead scraping them.

There aren't very many "control knobs" to affect the AI that I have found. I request the people who posted "it is so easy to figure out" to please share the "knobs" they have found.



It would be very helpful to have an autoplay feature. In other total conversion mods I have done, I have used this extensively to tune the AI behavior. There is no way to do this, or mod this in, with ES today.



I have done one experiment to improve AI behavior by reducing the number of "useless" and "unadvisable" buildings. In the one run I measured manually, this gave a significant improvement:



/#/endless-space/forum/27-general/thread/9289-suggestions-requested-rules-for-ai-to-not-build-something

When the decision was made not to allow AI to retrofit in previous patches (obviously now fixed)

I highly doubt they said, "Hey, let's prevent the AI from retrofitting". It was more like, "figuring out when/how to retrofit without wasting money is hard. Let's worry about that later."

Obviously if my field of study was Theology I could forgive you for calling me a douchebag for bringing it up.

If your field of study would have been Theology I would have been thrilled if not even in exctasy...



Right now I need someone with bit more humane interests to help with my mod

(then again I'm not sure whether Theologist could have answered what and how Amoebas see world)

Brainsqualling:



-AI uses one exploitation per system. Simple fix to make it choose optimum for planet. Best fix would be an early game dust/food specialization for all Arid, Tundra, Terran, Jungle, Oceans, and a switch to optimum as each advanced specialization is researched.

-AI does not stick to invading. It often moves invasion fleets around, delaying capture times. Make them keep one fleet on each enemy planet.

-AI has weak fleet building skills. Maybe the parameters for fleet grouping could be tweaked? Certainly want to prioritize Command Point techs more.

-AI needs to be able to choose a victory path and stick to it. This will require personality types for races. Not sure if it works here, but something to look into.

-AI needs to build better ships. Simple tweaks to ship designs will improve AI ships. Need to also increase certain weapon types and teach the AI to go after resources for weapons more aggressively.

-AI needs to value monopolies more and go to war to gain them.



I will be monkeying around with the xml today to see what is possible, but there is my shortlist of things I would like to help change. About me, I am an old GalCiv2 modder and have been working with Stardock's new game. If the AI is 10% of Fallen Enchantress' AI, most of these ideas should be easy.

@ seanw3, excellent list. I would add scrapping food improvements once pop cap is reached. I think we could probably debate optimal exploitation combinations. I claim your first one should always be food, second one should always be industry, and once you reach pop cap you should pick one focus for the system (either industry or science or dust) and switch all the exploits to that, unless you need a food to avoid starving.



Please do post if you are able to find controls for any of these in the xml, I have not found any of these.

AIParametersBuilding.xml uses FIDS+Approval+Diplomacy+Trade+Military to make construction decisions. Exploitations are set at value=2.1. Then 2.2 for the advanced, 2.3 for the third one. Can anyone figure out how the AI uses these values to actually make the construction decision? It could look for the system's best overall natural FIDS value or it might choose the specialization randomly.

The BailifDistrainee.xml might set the priority. Anyone know how it works?

BailiffDistrainee (some french words?) is used to set the priorities of the system governors. Each system governor will eventually build all the improvements given enough time, it is just a question of which order is used. See one detailed experiment here:



/#/endless-space/forum/27-general/thread/9523-the-tarkin-doctrine-experimenting-with-system-governors



The big thing which is missing is the global decision about whether a system builds an improvement or a ship. Also I do not "see" anything which describes how an exploit focus or governor is chosen for systems.



Can anybody find a hook which might allow different exploits in the same system, and/or, has anybody ever seen an AI system where different planets have different exploits?

Piecing together personalities, each race has a modifier to the standard bailiff values. Looks like the values are just building priority. The Bailiff sets the default priority. The race then modifies that to make sense for them. So each exploitation is valued after that.



Increasing the pass 0, 1, and 2 values for food and dust could be one way to help. I would like to try adding some AI value to each planet so the AI will choose the overall right exploitation. We might need new code to get each planet have the most efficient exploitation.

seanw3 wrote:
Best fix would be an early game dust/food specialization for all Arid, Tundra, Terran, Jungle, Oceans, and a switch to optimum as each advanced specialization is researched.

I've described a way more in-deptht way of how the AI should determine it's building projects several weeks ago.

The basic idea was to simulate the effect of each buildable improvement and divide the benefit by the required investment.

Food needs to be treated in a special way depending on how far away from maximum population the system is. The further away the higher the value of food compared to other fids-components.



I really doubt setting up these mechanisms is possible by just editing XML-files, though.



However, I think that by removing/altering "useless" improvements the building-AI can be brought up a lot.



I'm more concerned of how they use Heroes and how they decide when to move fleets though. A common observation was that they keep idle fleets in their back-systems even if their border-systems are under siege or they could take over systems.

And also the common use of non-maxed fleets is weird.



But count on me that I will dig into these XML-Files and look for possible parameters to improve the AI.

Ail wrote:
I've described a way more in-deptht way of how the AI should determine it's building projects several weeks ago.

I cannot find this thread very easily; can you give the link?

However, I think that by removing/altering "useless" improvements the building-AI can be brought up a lot.

I have done a lot of work on that here:

/#/endless-space/forum/37-modding/thread/15801-released-community-bug-fix-balance-mod

/#/endless-space/forum/27-general/thread/9289-suggestions-requested-rules-for-ai-to-not-build-something

If you can suggest any improvements, that would be great.

And also the common use of non-maxed fleets is weird.

Please retry with 1.0.14. I think the use of fleet spam and tiny fleets is entirely gone. This is a big step forward.

But count on me that I will dig into these XML-Files and look for possible parameters to improve the AI.

Great! Please post here, or in the modding forum with any findings.

I already found something that I think might be of great importance when it comes to what the AI builds.

It's only theories yet, so it might or might not be the way I think it is.



In AIParametersBuilding.xml each entry seems to stand for a specific building and has values assigned to it.

Example:

















These values match those in the Personalities-File and I guess they are multiplied and added up in order to decide what to build.

What I do not know, though is if the price of the building is factored in as well and what I'm pretty sure is not factored in is the amount of population affected for buildings that scale on population.



However, I'm pretty sure that by modifiying the values here will drastically affect the order in which the AI builds stuff. The one I took as example I think is the very cheap +6 to everything-Building. Now if I want the AI to build it top priority (since it is really cheap and I would build it top-priority) I think this would be done by just increasing the values here.



My suggestion would be to figure out which of those refers to what building and note the costs of them. Then take do something like 100/Production-cost and then multiply it with the values in the fields.



This should result in emphasizing cheaper buildings over expensive ones and probably not building certain too expensive-ones at all.



We can also factor in the Dust per Turn and reduce values accordingly to prevent the AI from building really expensive buildings too early.



Maybe the AI already multiplies the values found there with population, if they scale with it. Because I've observed them to build the flat-out-improvements like the 40 Research-lab with only very low priority. Thus this kind of buildings should probably have way better modifieres here in order for the AI to emphasize them as they should.



Another thing: Hero-Assignement of the AI is just stupid and now we know why:









1

Administrator;Corporate

Labor=6;Wit=6;Melee=3;Offense=0;Defense=0















So the AI is somehow forced to assign Hero01 to a fleet whereas we can see clearly is meant to be a System hero! Many other heroes suffer the same nonsensical fate.



The AIParametersHero.xml just needs to be redone to actually make sense and match the hero-types found in Hero.xml. Same goes for AiParametersHeroAbilities.xml... This file seems to determine how they distribute points when leveling up. And it's not matching the hero-types as well.



Edit:



Some assumptions I made were not true. The +6 to everything-building is "SystemBuildingSpecial1", not "StarSystemImprovementHabitat1". The AI neglects it so often because it only has 0.3 for each value.

I'm pretty sure the AI does not factor in if something depends on population or not. This is a drastic flaw when it comes to getting them to make the best possible decision what to built at a given time. Using the average population-number in these calculations might help as well.

However, we can now factor in Upkeep (can be found in StarSystemImprovementDescriptor) and Cost (can be found in StarSystemImprovement) into the values the AI uses and thus make it smarter in deciding what to build when.



Another thing, I suggest is to remove upkeep from all food-improvements so the inability to scrap them won't hurt the AI and it removes tedium from the player.



So my 1st goal is to fix hero-assignment. 2nd is coming up and applying an algorithm to the values within AIParametersBuilding.xml.

Ail wrote:
However, I'm pretty sure that by modifiying the values here will drastically affect the order in which the AI builds stuff.

Good find! This sounds pretty useful. I think it overlaps well with my changes to prevent some buildings from being built at all. Among buildings which can be built, it may be helpful to build some sooner, such as mag shields, planetary institute.

Another thing: Hero-Assignement of the AI is just stupid and now we know why

Another good find. This one does not seem to have quite as much effect as useless/unadvisable buildings. But, it cannot hurt to change it.

Another thing, I suggest is to remove upkeep from all food-improvements so the inability to scrap them won't hurt the AI and it removes tedium from the player.

This seems like a big change to play balance. If the only reason for this is so that "failure to scrap" will not hurt the AI, I would rather not do this.



If you have some changes done, you can certainly release them on your own; or if you prefer, I can roll the changes into my community mod; or both. Let us know if you have questions while implementing. The modding support is pretty new, and there are some ... quirks.



Community mod:



/#/endless-space/forum/37-modding/thread/15801-released-community-bug-fix-balance-mod



Protip on modding:



/#/endless-space/forum/37-modding/thread/16353-tutorials-for-modding-es

So mr. Engineer apparently you ain't did your homework. You should get rid of this PhD staff it aint suit you well.



The game engine is unity. Unity means c#. C# literally means open source. All gameplay related code is in Assembly-CSharp.dll file. google ILSPY tool.

Right now we are talking about xml changes only. Since you brought up the C# stuff, will that work with the current modding system? I like to keep things modular, so I would probably not go for anything that doesn't work with other mods. Messing with anything beyond the xml is out of my league. How about you guys?

What i am saying that all AI behavior hardcoded and modding is impossible. period. You can play with these meaningless numbers in xml but it changes almost nothing.