Hey Tychonoir!

SymbolStrings are simply "alternative short titles" for GuiElements. You can put anything you want in them.

However, most of them reference a GuiSpecialCharacter.

It is possible to create new GuiSpecialCharacters, but mind that these are "aliases" which simply replace a Key in any Localization text with a corresponding value.

Here, it replaces instances of "[pi]" with the symbol "π":

<GuiSpecialCharacter Codepoint="03C0" Key="[pi]" Value="π" />

The symbol must be in the game's font, and the value may be colored or anything you want (even text):

<GuiSpecialCharacter Codepoint="57369" Key="[foodColored]" Value="#b0c65d##REVERT#" />

It is useful when the key is referenced several times, and if the value is changed there it affects everywhere at once.

If you simply want to have something different in a GuiElement's SymbolString, you can actually put anything you want:

<SymbolString>[pi]</SymbolString>
<SymbolString>%PiTitle</SymbolString> (not 100% certain this one will work in every situation)
<SymbolString>PI but not localized</SymbolString>
<SymbolString>π</SymbolString>

What is the Codepoint field?

The CodePoint is not used in the code, that's... Unexpected but unsurprising.

Or... should I take this to mean the little icons (food, gears, etc.) are a part of the game's font, and there is no way to add new ones?

Unfortunately yes, you can't mix text and images, and the font is compiled in the game's resources and cannot be edited (I think it could but not without a lot of work).

However, you can look for symbols (in the font) that aren't used yet, and some of them may interest you.

Here are the ranges of symbols available in the game:

Default font (latin): Source Sans Pro
0400 - 04FF (cyrillic): Liberation Sans
4E00 - 9FFF (Chinese/Japanese/Korean): Noto Sans CJK
E019 - E484 (ES2): Amplitude font (with symbols like Cravers)

2500 - 254F (?): Noto Sans CJK
300A - 318F (?): Noto Sans CJK
AC00 - D7A3 (?): Noto Sans CJK
FF01 - 10106 (?): Noto Sans CJK
10000 - 15DBA (?): Noto Sans CJK

I think any character from the specified font in these ranges can be used, but characters outside of these are unavailable.